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Abstract 

The  focus  of  this  work  is  on  local  stability  of  a  class  of  nonlinear  ordinary  differential 
equations  (ODE)  that  describe  limits  of  empirical  measures  associated  with  finite-state 
exchangeable  weakly  interacting  IV-particle  systems.  Local  Lyapunov  functions  are 
identified  for  several  classes  of  such  ODE,  including  those  associated  with  systems 
with  slow  adaptation  and  Gibbs  systems.  Using  results  from  [5]  and  large  deviations 
heuristics,  a  partial  differential  equation  (PDE)  associated  with  the  nonlinear  ODE 
is  introduced  and  it  is  shown  that  positive  definite  subsolutions  of  this  PDE  serve  as 
local  Lyapunov  functions  for  the  ODE.  This  PDE  characterization  is  used  to  construct 
explicit  Lyapunov  functions  for  a  broad  class  of  models  called  locally  Gibbs  systems. 
This  class  of  models  is  significantly  larger  than  the  family  of  Gibbs  systems  and  several 
examples  of  such  systems  are  presented,  including  models  with  nearest  neighbor 
jumps  and  models  with  simultaneous  jumps  that  arise  in  applications. 
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1  Introduction 

In  this  paper  we  consider  local  stability  properties  of  the  nonlinear  ordinary  differen¬ 
tial  equation  (ODE) 

|p(t)=p(t)r(p(t)).  0.1) 

where  p(t)  takes  values  in  V(X).  Here  X  is  a  finite  set  that  we  denote  by  X  =  {1 , ,d}, 
V(X)  is  the  space  of  probability  measures  on  X  equipped  with  the  topology  of  weak 
convergence,  which  we  identify  with  the  unit  (d  —  1) -dimensional  simplex  S  =  {r  eHd  : 
rx  >  0,  x  E  X,  ^2xeX  rx  =  ^}  and  for  each  p  E  V(X),  T(p)  is  a  rate  matrix  for  a  Markov 
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chain  on  X .  Such  ODEs  describe  the  evolution  of  the  law  of  so-called  nonlinear  Markov 
or  McKean-Vlasov  processes  that  arise  as  limits  of  weakly  interacting  Markov  chains. 
Specifically,  consider  an  TV-dimensional  time-homogeneous  A'-valued  Markov  process 
XN  =  {X,,JV}j=ii...iAr,  where  for  t  >  0,  Xl’N (t)  represents  the  state  of  the  ith  particle  at 
time  t.  The  jump  intensity  of  any  given  particle  depends  on  the  configuration  of  other 
particles  only  through  the  empirical  measure 

1  N 

HN(t)  =  t  €  [0,oo),  (1.2) 

i=  1 

where  5a  is  the  Dirac  measure  at  a.  More  precisely  the  transitions  of  X N  are  determined 
by  a  family  of  matrices  {TN  {r)}rev(x)-  where  for  each  r  €  V(X),  TN(r)  =  {T^y(r),x,y  € 
if}  is  a  transition  rate  matrix  of  a  continuous  time  Markov  chain  on  X ,  in  the  following 
manner.  Given  XN(t)  =  x  €  XN ,  an  index  i  €  {1, . . . ,  N}  and  y  ^  Xi,  the  jump  rate  at 
time  t  for  the  transition 

{x\  i  •  •  *  i  Xi~\ ,  Xi-\- 1 , . . . ,  xn )  ^  (*ri  5  ■  ■  •  5  Xi~  i ,  //,  X{^\ , . . . ,  xn ) 

is  r xiy{rN {x)),  where  rN (x)  is  the  empirical  measure  given  as  r % ( x )  =  -T  J2iLi  1  {xi=z}' 
z  €  X .  The  jump  rates  for  transitions  of  any  other  type  are  zero.  If  converges  to 
a  Lipschitz  function  T,  uniformly  on  S,  and  pN(0)  converges  in  probability  to  q  £  V(X) 
then  /j,N  converges  uniformly  on  compact  time  intervals,  in  probability,  to  p(-)  given  as 
the  unigue  solution  of  (1.1)  with  initial  condition  q  (see  Theorem  2.2  of  [5]).  It  can  be 
shown  that  there  is  a  X -valued  stochastic  process  whose  law  at  time  instant  t  is  given 
by  the  solution  p(t)  of  (1.1)  and  the  ODE  (1.1)  is  referred  to  as  the  forward  equation 
of  this  nonlinear  Markov  process.  The  focus  of  the  current  paper  is  local  stability  (see 
Definition  2.3)  of  the  ODE  (1.1),  and  therefore  of  the  corresponding  nonlinear  Markov 
process,  for  several  families  of  models. 

As  usual  in  the  study  of  stability  of  dynamical  systems,  the  basic  approach  is  to 
construct  a  suitable  local  Lyapunov  function  (see  Definition  2.5).  It  is  known  that  for  an 
ergodic  linear  Markov  process  on  X  (i.e.,  the  case  where  T  is  constant)  the  mapping 
q  h->  R(q\\Tr),  where  R  is  relative  entropy  and  n  is  the  unique  stationary  distribution, 
defines  a  Lyapunov  function  for  the  associated  linear  Kolmogorov  equation.  The  proof 
of  this  result  uses  the  fact  that  if  p(t)  and  q(t)  denote  the  law  of  the  Markov  process 
with  two  different  initial  conditions,  then  the  forward  equations  for  p(-)  and  q(-)  are 
governed  by  the  same  fixed  rate  matrix  T(see,  for  example,  [12,  pp.  1-16-17]  and  [5, 
Section  3]).  For  nonlinear  Markov  processes,  since  the  rate  matrices  at  time  t  depend  on 
the  law  of  the  process  at  that  time  instant,  one  does  not  expect  this  property  to  hold  in 
general.  However,  in  Section  3  we  consider  a  family  of  models,  which  we  call  systems 
with  slow  adaptation,  for  which  relative  entropy  is  in  fact  a  Lyapunov  function  when 
the  adaptation  parameter  is  sufficiently  small.  This  result  says  that  relative  entropy 
continues  to  serve  as  a  Lyapunov  function  for  suitably  small  non-linear  perturbations  of 
linear  Markov  processes,  but  it  does  not  yield  Lyapunov  functions  for  general  nonlinear 
Markov  processes.  For  one  particular  family  of  models  whose  stationary  distributions 
take  an  explicit  form  and  which  we  call  systems  of  Gibbs  type.  Section  4  of  [5]  proposed  a 
candidate  local  Lyapunov  function  defined  as  the  limit  of  certain  scaled  relative  entropies 
that  involve  the  stationary  distributions  of  the  associated  A-particle  weakly  interacting 
Markov  processes.  Specifically,  denoting  the  unique  invariant  measure  of  the  AT-particle 
Markov  process  X N  by  ttn  €  V(XN),  the  candidate  Lyapunov  function  F  :  V{X)  — >  R  is 
defined  by 

F(q)  =  lim  FN(q)  =  lim  -jr=R  (®Nq\\nN)  ,  q  €  V(X).  (1.3) 

N—+OO  N — ^OO  iV 
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The  paper  [5]  shows  that  for  systems  of  Gibbs  type  the  function  defined  in  (1.3)  has 
a  simple  closed  form  expression.  In  Section  4  of  the  current  work  we  show  that  this 
function  is  in  fact  a  local  Lyapunov  function  in  the  sense  of  Definition  2.5  under  suitable 
positive  definiteness  assumptions. 

For  non-Gibbs  families,  stationary  distributions  usually  will  not  take  an  explicit  form 
and  thus  a  different  approach  is  needed.  One  such  approach  was  developed  in  Section  5 
of  [5],  where  instead  of  considering  limits  of  FN ,  we  consider  the  limits  of 

FtN(q)  =  ^ R(®Nq\\pN(t )), 

where  pN(t)  is  the  (exchangeable)  probability  distribution  of  XN (t)  with  some  exchange¬ 
able  initial  distribution  pN  { 0)  on  XN ,  as  JV  ->  oo  and  t  — »•  oo.  We  then  identify  the 
limit  as  N  — >  oo  of  FtN (q)  as  Jt{q),  where  Jt  is  the  large  deviations  rate  function  for  the 
collection  of  T^/Tj-valued  random  variables  {pN (t)}jveiN.  The  limit  of  Jt  as  t  — >  oo  was 
proposed  in  [5]  as  a  local  Lyapunov  function  for  the  ODE  (1.1),  though  the  question  of 
when  these  limits  exist  and  how  they  can  be  evaluated  was  not  tackled.  In  this  work 
we  approach  this  question  as  follows.  We  begin  by  formally  deriving  a  nonlinear  partial 
differential  equation  (PDE)  for  {Jt{q),t  >  0 ,q  €  <S}.  We  next  show  that  classical  sense 
positive  definite  subsolutions  of  the  stationary  form  of  the  PDE  (see  (5.7)),  which  is 
formally  the  equation  governing  the  limit  of  Jt  as  t  — >  oo,  are  local  Lyapunov  functions 
for  (1.1).  With  this  result,  the  problem  of  constructing  Lyapunov  functions  reduces 
to  finding  suitable  subsolutions  of  (5.7).  Although  finding  explicit  subsolutions  can  be 
challenging  in  general,  in  Section  6  we  introduce  an  interesting  family  of  models,  which 
we  call  locally  Gibbs  systems,  for  which  one  can  in  fact  give  an  explicit  solution  for 
(5.7).  As  shown  in  Section  6.1,  these  models  contain,  as  a  special  case,  the  Gibbs  type 
systems  studied  in  Section  4.  Moreover,  in  Sections  6.2  -  6.5  we  present  other  examples 
of  locally  Gibbs  systems,  including  models  with  nearest  neighbor  jumps  and  models  with 
simultaneous  jumps  that  arise  in  telecommunications  applications.  Finally  we  give  an 
example  to  illustrate  that  solutions  to  the  PDE  (5.7)  can  be  found  for  systems  that  are 
not  locally  Gibbs  as  well. 

The  paper  is  organized  as  follows.  Section  2  collects  some  definitions  and  basic 
results  related  to  stability  of  the  ODE  (1.1).  In  Section  3  we  study  systems  with  slow 
adaptation.  Section  4  considers  the  setting  of  systems  of  Gibbs  type.  We  then  study 
more  general  models  than  the  Gibbs  systems  of  Section  4.  In  Section  5,  we  present 
the  formal  derivation  of  a  nonlinear  time-dependent  PDE  that  is  satisfied  by  the  large 
deviation  rate  function  {Jt(q)}.  The  main  result  of  this  section  shows  that  a  positive 
definite  subsolution  of  the  stationary  version  of  this  PDE  is  a  local  Lyapunov  function  of 
(1.1).  Finally,  in  Section  6  we  identify  a  broad  family  of  models,  referred  to  as  locally 
Gibbs  systems,  for  which  a  non-trivial  subsolution  of  (5.7)  can  be  given  explicitly  and 
thus  under  suitable  additional  conditions  that  ensure  positive  definiteness,  one  can 
obtain  tractable  Lyapunov  functions  for  such  systems,  ensuring  local  stability.  We  also 
present  several  examples  that  illustrate  the  range  of  applicability  of  these  results. 

2  Local  Stability  and  Lyapunov  Functions 

In  this  section  we  will  collect  some  definitions  and  basic  results  related  to  stability 
of  the  dynamical  system  (1.1).  The  following  condition  will  be  assumed  on  several 
occasions. 

Condition  2.1.  The  function  p  T(p)  is  a  Lipschitz  continuous  map  from  S  to  R. 

Some  results,  such  as  the  main  result  of  this  section  (Proposition  2.6),  only  need  that 
T  be  continuous,  which  is  sufficient  to  ensure  the  existence  of  a  solution  for  any  initial 
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condition.  Denote  by  S°  the  relative  interior  of  S : 


S°  =  {p  £  S  :  pi  >  0  for  all  i  =  1, . . . ,  d}. 

We  first  recall  the  definition  of  a  locally  stable  fixed  point  of  an  ODE. 

Definition  2.2.  A  point  n*  €  S  is  said  to  be  a  fixed  point  of  the  ODE  (1.1)  if  the 
right-hand  side  of  (1.1)  evaluated  atp  =  t, r*  is  equal  to  zero,  namely, 

7r*r(7T*)  =  0. 


Definition  2.3.  A  fixed  point  n*  €  S°  of  the  ODE  (1.1)  is  said  to  be  locally  stable  if 
there  exists  a  relatively  open  subset  ID  of  S  that  contains  7r*  and  has  the  property  that 
whenever  p( 0)  €  ID,  the  solution  p(t)  of  (1.1)  with  initial  condition  p( 0)  converges  to  it* 

as  t  — »•  oo. 

Our  approach  to  proving  local  stability  will  be  based  on  the  construction  of  suitable 
Lyapunov  functions.  In  order  to  state  the  Lyapunov  function  property  precisely,  we  begin 
with  some  notation.  Let 

H  |  v  e  Bd  :  ^  Vi  =  1 

[  i= l 

be  the  hyperplane  containing  the  simplex  S,  and  let 

Ho  —  j  v  €  Ed  :  ^  Vi  =  0 
l  i= i 


be  a  shifted  version  of  this  hyperplane  that  goes  through  the  origin. 

GivenasetB  C  Hi,  a  function  U  :  ID  — >  R  will  be  called  differentiable  (respectively  C1) 
if  it  can  be  extended  as  a  differentiable  function  (respectively  continuously  differentiable 
function)  to  some  relatively  open  subset  ID'  of  Hi  such  that  ID  C  ID'.  In  particular,  for 
a  differentiable  function  U  on  a  relatively  open  subset  ID  of  Hi,  for  every  r  €  ID,  there 
exists  a  unigue  vector  D'mU(r)  €  Ho,  called  the  gradient  of  U  at  r,  such  that 

U(r  +  h)  —  U(r)  —  (Dt!mU (r),  h) 

hm  - — - =  0. 

heH0,  IN|->o  ||/i|| 


Note  that  if  {hi,  i  =  1, . . . ,  d  —  1}  is  an  orthonormal  basis  of  the  subspace  Ho,  we  can 
write 

d- 1 

D*°U (r)  =  ^(D*“(7(r),  hi) hi,  r  <=  B. 

i=i 

Finally,  we  say  that  the  differentiable  function  U  :  ID  — »  E  is  C1  if  the  mapping  r  h* 
Db"U(r)  from  B  to  Ho  is  continuous.  Freguently,  with  an  abuse  of  notation,  we  write 
DanU  simply  as  DU. 

We  introduce  the  following  notion  of  positive  definiteness. 

Definition  2.4.  Let  it*  €  5°  be  a  fixed  point  of  (1.1)  and  let  B  be  a  relatively  open 
subset  of  S  that  contains  i r* .  A  function  J  :  B  — »  B  is  called  positive  definite  if  for  some 
K*  €  E,  the  sets  Mk  =  {r  €  B  :  J(r)  <  K}  decrease  continuously  to  {tt*}  as  K  K* . 

In  Definition  2.4,  by  "decrease  continuously  to  {n*}"  we  mean  that:  (i)  for  every 
e  >  0,  there  exists  Ke  €  (K*,oo)  such  that  M k,  C  Be(7r*)  n  B,  where  Be(7r*)  is  the  open 
Euclidean  ball  of  radius  e,  centered  at  7r*,  and  (ii)  for  every  K  >  K* ,  there  exists  e  >  0 
such  that  Be(7r*)  n  S  C  MK.  Note  that  if  J  is  a  uniformly  continuous  function  on  B  which 
attains  its  minimum  uniquely  at  7r*  then  J  is  positive  definite.  A  basic  example  of  such  a 
function  is  the  relative  entropy  function  p  R{p\\-k*)  introduced  in  the  next  section. 
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Definition  2.5.  Let  tt*  G  5°  be  a  fixed  point  of  (1.1),  and  letD  be  a  relatively  open  subset 
ofS  that  contains  tt*  .  A  positive  definite,  C1  and  uniformly  continuous  function  J  :  B  — >  K 
is  said  to  be  a  local  Lyapunov  function  associated  with  for  the  ODE  (1.1)  if 

given  any  p{ 0)  €  ID.  the  solution  p(-)  to  the  ODE  (1.1)  with  initial  condition  p(0)  satisfies 
J ))  <  0  for  all  0  <  t  <  r  such  that  p(t)  ^  tt*  ,  where  t  =  inf{t  >  0  :  p(t)  G  IDe}.  In 
the  case  I)  =  <S°,  we  refer  to  J  as  a  Lyapunov  function. 

The  following  result  shows  that,  as  one  would  expect,  existence  of  a  local  Lyapunov 
function  implies  local  stability.  The  proof  is  standard,  but  is  included  for  completeness. 

Proposition  2.6.  Let  tt*  G  5°  be  a  fixed  point  of  (1.1)  and  suppose  that  Condition  2.1 
holds.  Suppose  there  exists  a  local  Lyapunov  function  associated  with  (]D,7r*)  for  (1.1) 
where  ID  is  some  relatively  open  subset  of  S  that  contains  tt*  .  Then  tt*  is  locally  stable. 


Proof.  Let  J  be  a  local  Lyapunov  function  associated  with  (B,7r*)  for  (1.1).  Since  J  is 
positive  definite,  there  exists  K*  G  K,  such  that  the  sets  Mk  =  {r  G  ID  :  J(r)  <  K} 
decrease  continuously  to  {7r*}  as  K  !  K* .  In  particular,  there  exists  L  G  (K* ,  oo)  and  a 
relatively  open  subset  B0  of  S  such  that  7r*  6  B0  C  ML  C  ID. 

We  will  prove  that  (1.1)  is  locally  stable  on  E>o,  namely 

whenever  p(0)  €  1D0 ,  the  solution  p(t)  of  (1.1)  converges  to  tt*  as  t  — >  oo.  (2.1) 

Note  that  (2.1)  is  clearly  true  ifp(0)  =  tt* .  Suppose  now  thatp(O)  ^  tt* .  If  p(t)  €  ID  then 

jtJW))  =  ( DJ(p(t)),p(t)V(p(t ))}. 

Let  t  =  inf{f  >  0  :  p(t)  €  1DC},  and  assume  that  r  <  oo.  Since  q  (D J (q) ,  qT (q)) 

is  a  continuous  function  on  ID  and  ^  J{p{t ))  <  0  whenever  p(t)  €  ID  \  {7r*},  we  have 
-$pJ{p{t))  <  0  for  all  t  €  [0,r).  Combining  this  with  the  fact  that  J  extends  continuously 
to  ID  due  to  uniform  continuity,  we  have  J(p(r))  <  L  and  consequently  p(r)  €  Ml  C  ID. 
This  contradicts  the  assumption  r  <  oo.  Hence  r  =  oo  and 

^ J(p(t ))  <  0  for  all  t  >  0  whenever  p(t)  ^  tt  * .  (2.2) 

Let  Kn  €  ( K*,L )  be  a  strictly  decreasing  sequence  such  that  Kn  \.  K*  as  n  — >  oo.  Let 

Tn  =  inf{f  >  0  :  p(t)  G  MKn }. 

Note  that  if  r„  <  oo,  then  p(t)  €  M k„  for  all  t  >  rn.  Since  the  sets  M k„  decrease 
continuously  to  {w*},  it  suffices  to  show  that  t„  <  oo  for  every  n. 

Consider  n  =  1.  If  p(0)  €  MKl,  n  <  oo  is  immediate.  Suppose  now  thatp(O)  0  MKl. 
Let  eo  >  0  be  such  that  Beo(7r*)  n  S  C  Mk1  C  Ml.  From  (2.2),  for  every  q  €  (BSo)c  n  Ml, 
^J(pq(t))\t=o  <  0  where  pq(t)  is  the  solution  of  (1.1)  with  p( 0)  =  q.  Recalling  the 
continuity  of  q  ( DJ(q ),  qT(q))  and  observing  that  (Beo)c  n  ML  is  a  closed  subset  of  ID 
we  have  that 

sup  (DJ(q),qT(q))  <  0. 

qe(BeoynML 

Also,  since  Beo(7r*)  fl  S  C  MKl,  for  all  t  <  T\,  p(t)  G  (Beo)c  n  ML.  Thus  we  have  that 
supt<Tl  ^  J{p{t ))  <  0.  This  shows  that  n  <  oo.  By  repeating  this  argument  we  see  that 
rn  <  oo  for  every  n,  and  the  result  follows.  □ 
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3  Systems  with  Slow  Adaptation 

Here  we  consider  the  case  where  the  ODE  (1.1)  exhibits  a  structure  we  call  slow 
adaptation,  for  which  the  strength  of  the  nonlinear  component  is  adjusted  through 
a  small  parameter.  The  long-time  behavior  of  systems  of  this  type,  in  the  context  of 
nonlinear  diffusions  arising  as  limits  of  weakly  interacting  Ito  diffusions,  is  studied  in 
[14]  based  on  coupling  arguments  and  hitting  times  (and  not  in  terms  of  Lyapunov 
functions). 

Suppose  that  Condition  2.1  holds  and  -k*  €  V(X)  is  a  fixed  point  of  the  ODE  (1.1). 
The  rate  matrix  TA(p)  =  T(A (p  —  n*)  +  n*)  corresponds  to  a  version  of  the  original  system 
but  with  slow  adaptation  when  A  >  0  is  small.  With  A  €  (0, 1]  fixed,  the  rate  matrices 
Tx(p),  p  €  V(X),  determine  a  family  of  nonlinear  Markov  processes.  The  corresponding 
forward  equation 

ftPX(t)=px(t)Tx(px(t))  (3.1) 

has  a  unique  solution  given  any  initial  distribution  p(0)  €  V(X).  Note  that  for  any 
A  €  [0, 1],  7 r*  is  also  a  fixed  point  for  (3.1).  We  are  interested  in  the  question  of  when  the 
fixed  point  7r*  is  locally  stable  for  sufficiently  slow  adaptation. 

Recall  that  given  p,  n*  6  V{X),  the  relative  entropy  of  p  with  respect  to  7r*  is  given  by 

R (pH71-*)  =  ^  px  !°g  ( •  (3.2) 

xex  x'Kx'' 

It  is  known  (see,  e.g.,  [12,  pp.  1-16-17]  or  [5,  Lemma  3.1])  that  the  mapping 

F(p)  =  R(p\\n*),  (3.3) 

serves  as  a  Lyapunov  function  for  finite-state  linear  Markov  processes.  The  forward 
equation  of  a  finite-state  linear  Markov  process  has  the  form  (1.1),  but  with  a  constant 
rate  matrix  T,  and  the  proof  of  the  Lyapunov  function  property  of  relative  entropy  for  such 
Markov  processes  crucially  uses  the  fact  that  T  is  constant.  In  contrast,  since  in  general 
the  rate  matrix  in  the  ODE  (1.1)  depends  on  the  state,  one  does  not  expect  7?,(-||7r*) 
to  serve  as  a  Lyapunov  function  for  general  finite-state  nonlinear  Markov  processes. 
Nevertheless,  in  this  section  we  will  show  that  for  systems  with  slow  adaptation  with 
A  sufficiently  small,  the  function  f?(-||7r*)  does  in  fact  have  the  desired  property.  The 
following  is  the  main  result  of  the  section.  Note  that  the  function  F  in  (3.3)  is  positive 
definite  (in  the  sense  of  Definition  2.4)  and  is  uniformly  continuous  on  5°.  Thus  the 
Proposition  below,  together  with  Definition  2.5,  says  that  F  is  a  Lyapunov  function 
associated  with  7r*  for  the  ODE  (3.1). 

Proposition  3.1.  Suppose  Condition  2.1  holds.  Letpx(-)  be  defined  by  (3.1)  and  F  by 
(3.3).  Suppose  that  T(7r*)  is  irreducible.  Then  there  is  \0  >  0  such  that  if  A  €  [0,  Ao],  then 
for  allt>0 

5hd«»<0, 

with  a  strict  inequality  if  and  onlyifpx(t)  7^  n*. 

Proof.  By  construction  and  hypothesis,  there  exists  C  €  (1,  00)  such  that  for  all  x,  y  €  X , 
all  A  >  0,  and  all  p  £  V(X), 


|rJ*(p)-rv*(0|  <AC||p-7rl, 

where  ||p  —  7r*||  =  J2xex  I  Px  —  7r*|.  Recall  that  since  7r*  is  stationary  7r*rA(7r*)  =  0.  Using 
the  definition  (3.2)  of  relative  entropy,  the  ODE  (3.1),  and  the  relation  yex  pyTyX(p)  = 
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J2x,yex  Py^yxi^*)  —  0, 

|k(^(«)ik-)  ,  £  Pirn  (log  (?|>) 


x,y£X 


+ 1 )  r Up  m 


=  Y  /'«(')  ( los 

x,y(E.X:x^y 


Px(t)*l 

Py(t)K 


Px(t)Ky 

Py(t)K 


+  1  V(7T*) 


+  ]T  p$(t)  log  (r^,(pA(f))  -  r,,(^)) 


x,y£X 


=  Y  p»w(los 

x,y£X:xy£y  \ 

+  H  PyWlog 

x,yE.X:x^y 


Px(t)K 

Py(t)< 

Px(t)K*y 

Py{t)K 


+  1  TVx{lC*) 


Px(t)Ky 

pHt)7 


(rA,(pA(  t))-TyX{n*)). 


where  we  use  the  convention  that  OlogO  =  0.  Fora:,?/  £  X  with  x  =/=■  y  andp  €  V(X),  set 

'PxK*v 


7 Vx{p)  =  Pv  log 


Pyl 


Py*i 


+1  UO, 


pyxip)  =  PyloS  iryx(p)  -  ryx(n*))  ■ 

\Pynx  / 

To  complete  the  proof  we  will  show  that  there  is  Ao  >  0  such  that  for  every  p  E  V(X), 
Y  byxip)  +  Pyxip))  <  0  for  all  A  £  [0,  A0],  (3.4) 

x,y€.X:x^y 

with  equality  if  and  only  if  p  =  7r* . 

It  is  straightforward  to  check  that  p  i-»  7 yx(p)  is  concave.  However  we  will  need  more 
than  that,  namely  a  uniform  estimate  on  its  second  derivative.  Let  r  £  Ho  with  ||r||  =  1. 
Evaluation  of  the  derivatives  gives 


d 


J~s  Y  'Yyxi'K*  +  sr) 


x,y£X:x^y 


=  0, 


s— 0 


ds2 


Y  7 yx(n*  +  sr) 

x,y£X:x^y 


Y 


x,y£JM:x^y 


rV(7r*).  (3.5) 


If  the  expression  in  (3.5)  is  zero  then,  since  7r*  >  0  for  all  y  and  all  states  communicate. 


for  all  x  pfp  y.  However  this  is  impossible,  since  r  £  Ho  requires  that  at  least  one 
component  be  of  the  opposite  sign  of  some  other  component.  Hence  the  expression  in 
(3.5)  is  negative.  Using  that  {r  :  ||r||  =  1}  is  compact  and  continuity  in  r  show  that  (3.5) 
is  in  fact  bounded  above  away  from  zero  on  this  set,  which  shows  the  matrix  of  second 
derivatives  is  negative  definite.  Using  the  fact  that  V(X)  is  compact,  we  find  that  there 
is  c  >  0,  not  depending  on  p,  such  that 

Y  7yz(p)  <  -C||P-  7T*||2, 

x,y£X:x^y 
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which  is  equivalent  to 

7y*(p)  <  ^l2  +  g  X]  7j/*(p)-  (3-6) 

x,y€.X:x^y  x,y£X:x^y 

Set  7min  =  min{r!/a:(7r*)  :  T^Tr*)  >  0}  >  0  and  note  that  maxxex  ^  <  00  since 
7r^in  =  min^g^  7r*  >  0.  Let  x,  y  £  X,  x  ^  y,  and  set  z  =  ^*7.  We  distinguish  two  cases. 

Case  1:  z>\  or  r  yx{p)  —  Fyx(n*)  >  0.  Suppose  first  thatpy  7^  0  and  z  >  1/2.  Since 
|  log  s|  <  2|s  —  1|  for  all  s  > 

Pyx(p)  =  Py  !°g  z(TyX(jp)  - 

rX(p)-r„(7T*)| 

=  Zi\Px^^PyK\\Tyx(p)  -  IW7r*)| 

**x 

<  Jr-  (n*y\Px  ~K\  +KK  -  Py\)  c'aIIp-7t*II- 

^"min 

This  inequality  is  trivially  true  if  py  =  0  or  if  a  <  1/2  and  Tyx(p)  —  ryx(-K*)  >  0,  and  thus 
is  always  valid  for  Case  1. 

Case  2:  Tyx(p)  -  Tvx(tt*)  <  0  and  z  €  [0,  |).  Since  log  s  —  s  +  1  <  0  for  all  s  >  0, 

h yx(p)+PyX(p)  =Py  (5  (logz-2  +  l)rV(7r*)  +logz(r^(p)  -Tyx(ir*))) 

<Py{h  0°S z-z  +  1)  7m;n  +  |  log z\2C\) 

<  \py  (-|  log (7min  -  ACX)  +  (1  -  z)7niin)  • 

This  quantity  is  non-positive  for  z  €  [0,  4)  whenever  A  <  Ai  =  /®g-Al.  Recalling  inequality 
(3.6),  we  have  for  A  €  [0,  Ai]  that 


< 


PxKv 


PvK 


hyx(P)  +  PyxiP)) 

x,y.x^y 

<-|l|p-7r*||2  +  ^^||p-7T*|[  {^ylPx-Kl+KK^Pyl) 

71111111  x,yeX:x^y 


<  ~\\\p-^*W2  +  ^r^\\p-n*\\  (  \Px-K\  +  \K~Py  1 

'‘min 


\.x£X 


yex 


,  cn  *  11 2  .  4CA  ||  2 

<-oll P-n  II  +  3HI P-n  II  • 

2  ^min 


This  last  quantity  is  strictly  negative  if  A  <  A2  =  min{Ai,  and  p/  7r*,  and  zero  for 

p  =  7r*.  Choosing  Aq  €  (0,  A2),  we  find  that  (3.4)  holds,  with  equality  if  and  only  if  p  =  7r*. 


The  bound  on  A  obtained  in  the  proof  is  obviously  conservative,  and  better  bounds 
that  depend  on  T(7r*)  and  tt*  can  be  found. 


4  Systems  of  Gibbs  Type 

In  this  section  we  revisit  the  class  of  Gibbs  models  introduced  in  Section  4  of  [5].  We 
begin  by  recalling  the  basic  definitions.  Let  K  :  X  x  K,d  — >  K  be  such  that  for  each  x  €  X, 
K(x,  •)  is  a  continuously  differentiable  function  on  Rri.  We  sometimes  write  K(x,p)  as 
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Kx(p).  One  special  case  we  discuss  in  detail  is  given  by 

K(x,p)  =  V(x)  +  p'52w(x,y)pvt  (x,p)eXxM.d  (4.1) 

vex 


where  V  :  X  — »R,  W  \  X  x  X  and  /3  >  0. 

Let  ( a(x ,  y))x, yex  be  an  irreducible  and  symmetric  matrix  with  diagonal  entries  equal 
to  zero  and  off-diagonal  entries  either  one  or  zero.  Define  H  :  X  x  Kd  — >  R  by 


H(x,p)  =  Hx(p)  =  Kx(p)  + 


d 

dpx 


Pz 


(4.2) 


and  I'lLxLxtf-fRby 

*(*,», p)  =  Hu(p)  -  Hx(p ),  (x,y,p)  €  X  x  X  x  Rd. 


Let 

r x,y(p)  =  fi-(*(*.w.P))+a(x,  y),x^y,pe  V(X),  (4.3) 

where  recall  that  we  identify  V(X)  with  the  simplex  S.  Then  for  p  €  5,  T(p)  is  the 
generator  of  an  ergodic  finite-state  Markov  process,  and  the  unique  invariant  distribution 
on  X  is  given  by  n (p)  with 

7T (p)x  =  irrx  exp  j  (4-4) 

zm 

where 

Z(P)=J2^P(-HX(P))- 

xex 

By  studying  the  asymptotics  of  certain  scaled  relative  entropies,  the  following  candi¬ 
date  Lyapunov  function  was  identified  in  Theorem  4.2  of  [5]: 

F(p)  =  (RX(p)  +  lo&Px)Px  (4.5) 

x£X 

for  p  €  <S.  We  note  that  in  [5]  K(x,  •)  was  taken  to  be  twice  continuously  differentiable 
(this  property  was  used  in  the  proof  of  Lemma  4.1  of  [5]),  however  here  we  merely 
assume  that  K(x,  •)  is  C1 .  Also  note  that  in  the  special  case  of  (4.1), 


f(p)  =  ^2  I  v(x)  +  P^2w{x,y)py  +  \ogpx\'Px-  (4.6) 

xex  \  yex  ) 

Since  J2xexPx  log  px  is  the  negative  of  the  entropy  ofp,  F  is  the  sum  of  a  convex  function, 
an  affine  function  and  a  quadratic  function  on  V(X).  This  fact  is  useful  in  determining 
whether  or  not  the  fixed  points  of  (1.1)  are  stable. 

Recall  the  set  7iQ  introduced  in  Section  2  and  note  that  for  p  €  5°  =  {p  €  S  :  px  > 
0  for  all  x  =Hl, . . . ,  d}  the  directional  derivative  of  the  function  F  in  (4.5)  in  any  direction 
v  €  Ho  is  given  by 

F~f(p)  =  (df(p)’v)  =  Vx  I  logPx  +  K*(p)  +  PyF—KV(P)  )  ■  (4-y) 

xex  y  yex  Px  ) 

where  we  have  used  that  Yhxex  vx  =  0.  The  following  result  shows  that  the  fixed  points 
of  (1.1)  can  be  characterized  as  critical  points  of  F. 
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Theorem  4.1.  Let  T  be  as  defined  in  (4.3)  and  p  £  V{X).  Then  p  is  a  fixed  point  for  (1.1) 
if  and  only  if  p  £  S°  and  J^F(p)  =  0  for  all  v  £  Ho- 

Proof.  Recall  that  tt (p)  is  the  unique  invariant  probability  associated  with  T(p),  and 
hence  jr(p)r(p)  =  0.  Also  note  that  p  is  a  fixed  point  for  (1.1)  if  and  only  if  pT(p)  =  0, 
which,  since  T(p)  is  a  rate  matrix  of  an  ergodic  Markov  process,  can  be  true  if  and  only 
if  p  =  7 r(p).  Since  ir(p)  £  5°  for  every  p  £  S  we  have  that  any  fixed  point  of  (1.1)  is  in  5°. 
For  x,  y  £  X,  x  ^  y,  let  vx,v  =  ex  —  ey,  where  ex  is  the  unit  vector  in  direction  x.  Then  by 
(4.7),  (4.2)  and  (4.4),  for  anyp  £  S° 


dvx>y 


F(p)  =  log  px  -  log  py  +  Kx(p)  -  Ky(p) 


+  E 

z€X 

=  log  px  -  log  py  +  (. Hxip )  -  Hv(p)) 
'n(p) 


=  log  -  log 


Tr(p) 


(4.8) 


If  p  is  a  fixed  point  of  (1.1)  then  p  =  % (p),  and  so  F(p )  =  0  for  all  x,  y  £  X,  x  ^  y. 
From  this  it  follows  that  J^F(p)  =  0  for  all  v  £  TLq. 

Conversely,  suppose  p  £  <S°  and  J^F(p)  =  0  for  all  v  £  TLq.  Then  from  (4.8) 


Px 

Py 


n(jp)x 

n(p)y 


for  all  x,  y  £  X . 


Thus  p  =  7 r(p),  which  says  thatp  is  a  fixed  point  of  (1.1). 


□ 


According  to  Theorem  4.1,  the  equilibrium  points  of  the  forward  equation  (1.1)  are 
precisely  the  critical  points  of  F  on  V(X).  Note  that  although  for  each  p,  T(p)  is  a  rate 
matrix  of  a  Markov  process  with  a  unique  invariant  measure,  the  dynamical  system  (1.1) 
can  have  multiple  stable  and  unstable  equilibria.  Here  is  an  example. 

Example  4.2.  Assume  that  X  =  {1,2},  and  K  is  given  as  in  (4.1)  with  V  =  0,  W{  1, 1)  = 
0  =  W{ 2,2),  and  W(  1,2)  =  1  =  W{ 2, 1).  Then  F(p)  =  f{Pl)  with 

f(x)  =  xlogx  +  (1  —  x)  log(l  —  x)  +  2/3(1  —  x)x,  x  £  [0, 1]. 

The  critical  points  of  F  on  ^({l,  2})  are  in  a  one-to-one  correspondence  with  the  critical 
points  of  f  on  [0, 1],  We  have  /( 0)  =  0  =  /( 1),  and  for  x  £  (0, 1) 

f{x)  =  log  x  -  log(l  -x)  +  2/3-  4/3®,  f"(x)  =  -  +  — 1 - 4/3. 

x  1  —  X 

Moreover,  f(x)  —oo  as  x  tends  to  zero,  and  f'(x)  — >  oo  as  x  tends  to  one. 

If  /3  <  1  then  f  has  exactly  one  critical  point,  namely  a  global  minimum  at  x  =  {.  If 
/3  >  1  then  there  are  three  critical  points,  one  local  maximum  at  x  =  \  and  two  minima 
at  xp  and  1  —  xp,  respectively,  for  some  xp  £  (0,  }),  where  xp  I  as  /3  1 1,  xp  0  as 
/3  goes  to  infinity.  The  two  minima  of  f  correspond  to  stable  equilibria  of  the  forward 
equation,  while  the  local  maximum  corresponds  to  an  unstable  equilibrium. 

4.1  Lyapunov  function  property 

Suppose  that  the  function  F  defined  in  (4.5)  is  positive  definite  (in  the  sense  of 
Definition  2.4)  in  a  neighborhood  of  a  fixed  point  n*  of  (1.1)  which  contains  no  other 
fixed  point  of  (1.1).  In  this  section  we  show  that  F  is  a  local  Lyapunov  function  for  the 
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ODE  (1.1)  (associated  with  the  neighborhood  and  the  fixed  point  %*),  with  T  defined  by 

(4.3) .  This  result  is  an  immediate  consequence  of  the  theorem  below  and  Definition  2.5. 
Together  with  Proposition  2.6  this  will  imply  i r*  is  locally  stable. 

Theorem  4.3.  Letp(-)  be  a  solution  to  the  forward  equation  (1.1)  with  T  as  defined  in 

(4.3)  and  some  initial  distribution  p(0)  €  V(X).  Then  for  all  t  >  0, 


inm) 


jtR{p(f)h{q)) 


<  o. 

q=v(t) 


Moreover,  ^ F(jp(t ))  =  0  if  and  only  if  p(t)  =  jr(p(t)). 


(4.9) 


Proof.  We  will  show  that  if  p(-)  is  the  solution  to  (1.1)  with  p(0)  =  q  then 


JtF «*» 


t=0 


jtR(p(t)\\-K(q)) 


(4.10) 


In  view  of  the  semigroup  property  of  solutions  to  the  ODE  (1.1),  and  since  q  is  arbitrary, 
the  validity  of  (4.10)  implies  the  first  equality  in  (4.9). 

Let  p(0)  =  q.  By  the  definition  (4.5)  of  F  and  since  J2xex  ^-(0)  =  0, 


JtF <**» 


t=0 


Y Iog  9x^(0)  +  Y 1<x (9) 

x£X  x€.X 


dpx 

dt 


(»)+  E 

x.yEX  y 


dPy 

dt 


(0). 


On  the  other  hand,  by  the  definition  of  relative  entropy,  (4.4)  and  (4.2),  and  again  using 
the  relation  J2xex  ^  (0)  =  0,  we  have 


J?(p(f)||7r(g)) 


t= 0 


dt 


Px{t)  log px(t) 


xex 


=  Y  log  9*^(0)  +  A'a'(9)“^(0) 


-  Y  -^f(0)log7r^(9) 

t= 0  X^x 
dpx 


(4.11) 


xex 


dt 


xex 


dt 


X,zex 


'  dp  .. 


dt 


Comparing  the  right  sides  of  the  last  two  displays  we  see  that  (4.10)  holds. 

The  rest  of  the  assertion  follows  from  the  observation  that  ir(q)  is  the  stationary 
distribution  for  the  (linear)  Markov  family  associated  with  T(q)  and  from  the  Lyapunov 
property  of  relative  entropy  in  the  case  of  ergodic  (linear)  Markov  processes;  see  Lemma 
3.1  in  [5],  □ 


Remark  4.4.  Consider  the  slow  adaptation  setting  of  Section  3  for  the  Gibbs  model 
with  K  as  in  (4.1).  Thus  we  start  from  a  family  of  rate  matrices  T(p),  p  €  V(X),  defined 
according  to  (4.3).  Suppose  that  tt*  is  a  fixed  point  of  the  mapping  p  n(p).  For 
X  €  [0, 1],  p  €  V(X),  set  Tx(p)  =  r(7T*  +  X(p  —  n*)).  The  rate  matrices  TA(p)  are  again  of 
Gibbs  type ,  that  is,  TA(p)  satisfies  (4.3),  but  with  replaced  by  'kA,  where  'kA  is  defined 
exactly  as  41  is  with  K  as  in  (4.1),  but  with  different  potentials  in  place  of  V  and  W.  In 
particular,  the  potentials  Vx’13,  Wx  are  given  by 

VX’P{x)  =  V(x)  +  2/5(1  -  A)  Y  W(x’ z K>  Wx{x,  y)  =  A W(x,  y). 

zex 
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Fix  A  >  0.  Then  (4.6)  and  Theorem  4.3  imply  that  if  the  function 

FXiP)  ~YPx  log px  +  Y  VX’P(x)px  +P  Y  W^A(a;.  V)PxPy 

x^X  x£X  x,y£X 


+  A^  Y  W(x>y)PxPy , 

x,yEX 


is  positive  definite  in  some  neighborhood  of  n*,  then  it  is  a  local  Lyapunov  function  for 
(3.1)  (associated  with  that  neighborhood  and  the  fixed  point  n*).  Proposition  3.1,  on 
the  other  hand,  implies  that  F(p)  =  R  (p||7r*)  is  also  a  local  Lyapunov  function  when  A  is 
positive  but  sufficiently  small.  By  the  definition  of  relative  entropy,  (4.4),  (4.2)  and  (4.1), 

F(p)  =  R(p\\ir*) 

=  Y Px  logpx  -  Px  log  nx 

xEX  xEX 

=  YPxlogPx  +I°Sz(7r*)  +  Y  \  V(x)  +  2PY  w(x,z)tt*\  px, 

xex  xex  V  zex  ) 

which  is  equal  to  Fx(p)  +  logZ(7r*)  for  A  =  0.  Observe  that  the  term  logZ(7r*)  has  no 
impact  on  the  Lyapunov  function  property  as  it  does  not  depend  on  p.  Thus,  the  function 
Fx  includes  "correction  terms"  (that  vanish  when  A  =  0)  and  selves  as  a  Lyapunov 
functionfwhen  positive  definite)  not  just  for  small  A  but  rather  for  all  A  €  (0, 1]. 

4.2  Comparison  with  existing  results  for  Ito  diffusions 

A  situation  analogous  to  that  of  this  section  is  considered  in  [13],  where  the  author 
studies  the  long-time  behavior  of  "nonlinear"  Ito-McKean  diffusions  of  the  form 

dX(t)  =  -  (vV(X(t))  +  2(3  J  X{W(X{t),y))pt{dy)\dt  +  y/2dB(t),  (4.12) 

where  pt  is  the  probability  law  of  X(t),  B  is  a  standard  d-dimensional  Wiener  process,  V 
a  function  Kd  h*  1R,  the  environment  potential,  and  W  a  symmetric  function  Rd  x  IRd  h->  R. 
with  zero  diagonal,  the  interaction  potential.  Here  Vi  denotes  gradient  with  respect 
to  the  first  Revalued  variable.  Signs  and  constants  have  been  chosen  in  analogy  with 
the  finite-state  models  considered  here.  Solutions  of  (4.12)  arise  as  weak  limits  of  the 
empirical  measure  processes  associated  with  weakly  interacting  Ito  diffusions.  The 
Af-particle  model  is  described  by  the  system 

2/1  N 

dXl'N(t)  =  —W(Xl’N(t))dt  —  YYVl  w(xt'N  (t),x3,N  +  V2dBt(t), 

V  i=i 

where  i  €  {1, . . . ,  N},  B1, . . . ,  BN  are  independent  standard  Brownian  motions. 

In  [13]  a  candidate  Lyapunov  function  F :  P(Rd)  — >  [0,  oo],  referred  to  as  the  "free 
energy  function",  is  introduced  without  explicit  motivation,  and  then  shown  to  be  in  fact 
a  valid  Lyapunov  function.  The  same  function  is  also  considered  in  [6]  and  plays  a  key 
role  in  their  study  of  convergence  properties  of  gt  as  t  — >  oo.  The  function  takes  the 
following  form.  If  g  is  a  probability  measure  that  is  absolutely  continuous  with  respect 
to  Lebesgue  measure  and  of  the  form  fll(x)dx,  then 

F{p)  =  J  log  f^f^dxF  J  V(x)p(dx)  +  j  j  W(x,y)p(dx)p(dy).  (4.13) 
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In  all  other  cases  F(fi)  =  oo.  This  function  is  clearly  a  close  analogue  of  the  function  in 
(4.5),  which  was  derived  as  the  limit  of  scaled  relative  entropies.  There  are,  however, 
some  interesting  differences  in  the  presentation  and  proof  of  the  needed  properties.  The 
most  significant  of  these  is  how  one  represents  the  derivative  of  the  composition  of  the 
Lyapunov  function  with  the  solution  to  the  forward  equation.  In  [13]  the  descent  property 
is  established  by  expressing  the  orbital  derivatives  of  F  in  terms  of  the  Donsker-Varadhan 
rate  function  associated  with  the  empirical  measures  of  solutions  to  (4.12),  when  the 
measure  pt  is  frozen  at  p  €  7,(IRd).  In  contrast,  in  our  case  the  orbital  derivative  of  the 
Lyapunov  function  is  expressed  as  the  orbital  derivative  of  relative  entropy  with  respect 
to  the  invariant  distribution  n(p)  that  is  obtained  when  the  dynamics  of  the  nonlinear 
Markov  process  are  frozen  at  p.  The  latter  expression  also  applies  to  the  diffusion  case 
in  the  sense  that  for  all  t  >  0, 

±F^t)  =  ±R(pt\K)^t,  (4.14) 

where  pt  is  the  law  of  X(t),  X  being  the  solution  to  (4.12)  for  some  (absolutely  continu¬ 
ous)  initial  condition,  and  7r„  €  V(M,d)  is  given  by 

ir„(dx)  =  exp  (■ -V(x)  -  2/3  [  W(x,y)v(dy) )  dx,  (4.15) 

\  Jnd  ) 

with  Zv  the  normalizing  constant.  Clearly,  the  probability  measures  given  by  (4.15) 
correspond  to  the  distributions  ir(p)  €  V(X)  defined  in  (4.4).  The  relationship  (4.14) 
can  be  established  in  a  way  analogous  to  the  proof  of  Theorem  4.1.  On  the  other  hand, 
the  representation  for  the  orbital  derivative  of  the  Lyapunov  function  in  terms  of  the 
Donsker-Varadhan  rate  function  as  established  in  [13]  for  the  diffusion  case  does  not 
carry  over  to  the  finite-state  Gibbs  models  studied  above.  Because  of  this,  we  argue  that 
(4.14)  is  the  more  natural  and  general  way  to  demonstrate  that  F  has  the  properties 
required  of  a  Lyapunov  function. 

To  make  this  more  precise,  consider  the  case  of  linear  Markov  processes.  Let  T  be  the 
infinitesimal  generator  (rate  matrix)  of  an  X -valued  ergodic  Markov  family  with  unique 
stationary  distribution  ix  €  V{X).  Let  p(-)  be  a  solution  of  the  corresponding  forward 
equation  (1.1).  Then 

^-R(pWIK)  =  J  ftT  (log  ft)  dir  =  -£r(/t,log/t), 

where  ft  =  is  the  density  of  p(t)  with  respect  to  7r  and  •)  is  the  Dirichlet  form 
associated  with  V  and  its  stationary  distribution  n,  that  is. 


£r (f,g)  =  -  j  /r(p)d7r  =  -  f(x)  ( 

Jx  x£X  \y£X  J 


for  test  functions  /,  g  :  X  — >•  IR,.  On  the  other  hand,  the  Donsker-Varadhan  /-function 
associated  with  T  is  given  by 


Mm) 


sup 

f-.X-t(  0,oo) 


r(/)Qr) 

/(*) 


p€V(X). 


If  T  is  also  reversible  (i.e.,  Vxvt:x  =  TyxTry  for  all  x,  y  €  X),  then  I  takes  the  more  explicit 
form 


(  /dp 

Idp' 

IV  dn'\ 

1  dir  , 
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see,  for  instance.  Theorem  IV.14  and  Exercise  IV.24  in  [10,  pp.  47-50].  In  general,  the 
functions  /  £v  (,//,  s/f'j  and  /  £v  (/,  log  /)  with  /  ranging  over  all  non-degenerate 
7r-densities  are  not  proportional.  As  a  counterexample,  it  is  enough  to  evaluate  the 

Dirichlet  forms  for  T  =? . /  ^  and  7 r  =  (1/2, 1/2). 

5  A  PDE  for  Limits  of  Relative  Entropies 

In  the  last  section  we  saw  that  the  scaling  limits  of  relative  entropies  with  respect 
to  stationary  distributions  of  certain  /V-particle  Markov  processes  XN  yield  candidate 
Lyapunov  functions  for  (1.1).  In  this  section  we  consider  the  case  where  closed  form 
expressions  for  the  stationary  distributions  are  not  available  and  consequently  these 
limits  cannot  be  evaluated  explicitly.  Recall  from  the  discussion  in  Section  5  of  [5]  that 
in  such  cases  our  basic  approach  to  constructing  Lyapunov  functions  is  to  take  limits 
of  the  scaled  relative  entropy  F/v  specified  in  equation  (1.4)  of  [5]  (see  also  equation 
(5.5)  in  this  section),  first  asJV^oo  and  then  as  t  — >•  00.  Theorem  5.5  of  [5]  shows  that 
under  some  basic  assumptions,  the  limit  as  N  — 7  00  coincides  with  the  large  deviation 
rate  function  Jt(-)  :  S  —7  [0,  00)  for  a  certain  sequence  of  empirical  measures  of  N- 
particle  systems  that  converge  to  the  solution  of  the  ODE  (1.1).  This  large  deviations 
result[ll,  4,  8]  is  recalled  in  Section  5.1.  Next,  we  formally  derive  a  time-dependent 
PDE  for  the  associated  large  deviation  rate  function  Jt(r)  in  Section  5.2,  and  present  the 
stationary  version  of  this  PDE  in  Section  5.3.  These  formal  calculations  simply  motivate 
the  form  of  the  PDE  -  the  main  result  presented  in  Section  5.5  that  subsolutions  to  the 
stationary  PDE  serve  as  local  Lyapunov  functions  for  the  ODE  (1.1),  does  not  rely  on 
this  derivation.  The  proof  of  the  main  result  relies  on  certain  properties  that  are  first 
established  in  Section  5.4. 


5.1  A  large  deviation  result 

Let,  as  in  Section  2  of  [5],  XN  =  (X1,N , ....  XN,N)  be  a  T'JV-valued  Markov  process 
with  transitions  governed  by  the  family  of  matrices  {T(r),r  €  P(X)},  where  for  each 
r  €  V(X),  T(r)  =  {Txy(r),x,y  €  X}  is  a  transition  rate  matrix  of  a  continuous  time 
Markov  chain  on  X  (here  for  simplicity  we  assume  that  rw  =  T).  Specifically,  the 
transition  mechanism  is  as  follows.  Given  XN  (t)  =  x  e  XN ,  an  index  i  e  {1, . . . ,  N}  and 
y  7^  x,,  the  jump  rate  at  time  t  for  the  transition 


{x\  ,  .  .  .  ,  Xi—  1 ,  Xj-f-  1 ,  .  .  .  ,  Xn  )  1  ^  i_Xl ,  .  . 


Xi—  1 ,  7/,  Xj-f-l ,  .  .  .  ,  Xn  ) 


is  rXiV(rN(x)),  where 


<(*) 


(5.1) 


The  jump  rates  for  transitions  of  any  other  type  are  zero.  Under  the  assumption  of 
exchangeability  of  the  initial  random  vector  {X*’Ar(0)}i=i  n  we  have  that  the  processes 
_ _ /v  are  also  exchangeable.  From  this,  it  follows  that  the  empirical  measure 
process  yN  =  is  a  Markov  chain  taking  values  in  Sm  =  S  fl  jj^d,  where  S  is 

the  unit  simplex  which  is  identified  with  'P(X),  with  the  generator  CN  given  by 


£Nf(r)  =  E 

x,y£X:x^y 


NrxTxy( 


r+^(ey^ex 


f(r 


(5.2) 


for  real-valued  functions  /  on  Sat. 

We  recall  the  following  locally  uniform  LDP  for  the  empirical  measure  process.  The 
LDP  has  been  established  in  [11,  4]  while  the  locally  uniform  version  used  here  is  taken 
from  [8]. 
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Theorem  5.1.  Suppose  that  for  each  p  £  S,  T(p)  is  the  transition  rate  matrix  of  an 
ergodic  Markov  chain  and  that  Condition  2.1  holds.  Fort  €  [0,oo)  let  pN  (t)  be  the 
distribution  of  XN (t)  =  (X1,N  (t), . . . ,  XN,N(t)).  Recall  the  mapping  rN  :  XN  — >  Pn{X) 
given  by  (5.1),  i.e.,  rN(x)  is  the  empirical  measure  of  x  .  Assume  that  the  initial  random 
vector  i’JV(0)}i=ij...jAr  is  exchangeable  and  assume  that  rN  under  the  distribution 
pN( 0)  satisfies  a  large  deviation  principle  (LDP)  with  a  rate  function  .J0.  Then  for  each 
t  €  [0,  oo),  rN  under  the  distribution  pN(t)  satisfies  a  locally  uniform  LDP  on  V(X)  with 
a  rate  function  .Jt,  thus  given  any  sequence  {qatIatsm/  Qn  €  5  at,  such  that  qN  — >  q  €  S, 

lim  4  log pN(t)  ({ y  €  XN  :  rN(y)  =  qN})  =  -Jt{q)- 

N — yoo  iv 

Furthermore,  Jt(q )  <  oo  for  all  q  €  V(X). 

We  will  now  formally  derive  a  PDE  solved  by  Jt(q). 


5.2  A  time-dependent  PDE 

For  notational  convenience,  throughout  this  section  for  t  >  0  and  r  €  S  we  write  Jt(r) 
as  J(r,t).  For  t  >  0,  let  uN (t)  denote  the  distribution  of  pN{t),  that  is,  for  r  €  Sn,  let 
u^f  ( t )  =  P  (gN(t)  =  r).  Then,  uN  satisfies  the  Kolmogorov  forward  equation 

^-(t)  =  uN  {t)CN ,  (5.3) 

where  CN  is  as  in  (5.2).  For  r  €  Sn,  substituting  into  (5.3)  the  approximation 

(f)  «  e~NJ^ 


that  follows  from  the  LDP  stated  in  Theorem  5.1,  and  recalling  the  form  of  the  generator 
CN  from  (5.2),  we  obtain 

=  £  e-JVJ(r-i(eB-a,),i)  +  1}  ^  L.  _  I(6y  _  eE 

x,y€X,x^y:  '  ' 

r- Xr(ey~ex)^SN 

Y  e~NJ^NrxVxy(r). 

x,y€X  ,Xy6y. 
r+w(ev~eAsSN 


Observing  that  the  left-hand  side  of  the  last  display  equals  —Ne  NJ(r’t')  ( dJ(r ,  t)/dt),  and 
multiplying  both  sides  by  eNJ(r,t')  /N,  we  obtain 


d 

dt 


J (r,  t) 


E 


e-N[j(r-^(ey-ex),t)-J(r,t)\ 


X,yex  ,Xy6y. 

E  rxTxy(r). 


x,yeX,xj±y. 

r+jf(ey—ex)eSN 


If  J  is  smooth  and  N  is  large,  we  can  use  the  approximation 

J  (r  ~  -^{ev  -  ex),tj  ~  J(r,t)  «  ~^( DJ(r,t),ey  -  ex)+o(l/N). 

Substituting  this  approximation  into  the  previous  display,  sending  N  — >  oo  and  recalling 
that  rXi!/(-)  is  continuous,  we  obtain  the  following  PDE  for  J(r,  t ):  for  r  €  S  and  t  €  [0,  oo), 

-^(r,t)  =  E  [e(DJ^ev^  - 1]  rxTxy(r).  (5.4) 

x,y£X:x^y 
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As  mentioned  earlier,  this  derivation  is  not  rigorous  because  we  did  not  establish  the 
smoothness  properties  of  J  assumed  in  the  calculations.  In  fact,  in  general  one  does  not 
expect  this  smoothness  property  to  hold  and  one  would  have  to  interpret  J  as  a  viscosity 
solution  to  the  PDE  (5.4)  with  appropriate  boundary  conditions.  However,  the  derivation 
simply  serves  to  motivate  the  form  of  the  stationary  PDE  and  the  proof  of  the  main  result 
given  in  the  next  section  does  not  rely  on  this  derivation. 

5.3  The  stationary  PDE  and  Lyapunov  functions 

We  now  introduce  our  main  tool  for  constructing  local  Lyapunov  functions  for  (1.1). 
Recall 

FtN(q)  =  R(®Nq\\pN(t )),  q  G  5.  (5.5) 

Formally  writing 

lim  lim  FtN (q)  =  lim  Jt(q)  =  J*(q),  q£S,  (5.6) 

£->-oo  N— >oo  t— >oo 

one  expects  from  the  formal  derivation  of  the  last  section  that  the  function  J*  solves  the 
following  PDE: 

H(r,  —DJ(r))  =  0,  (5.7) 

where  for  (r,  a)  G  S  x  Rd, 

H(r,a)  =  —  J2  rxVxv{r)[e-^ey-^  -l\  (5.8) 

x,yEX:x^y 

(we  use  H  to  distinguish  from  H(x,p)  as  used  in  the  section  on  systems  of  Gibbs  type). 
In  the  introduction  of  [5]  it  was  discussed  why  the  limit  function  J*  may  serve  as  a 
(local)  Lyapunov  function  for  (1.1).  This  suggests  solutions  of  the  stationary  PDE  (5.7)  as 
candidates  for  a  Lyapunov  function.  The  main  result  of  this  section  makes  this  precise 
by  proving  that  positive  definite  subsolutions  of  (5.7)  give  local  Lyapunov  functions  for 
(1.1).  To  state  the  precise  result  we  begin  by  recalling  the  definition  of  a  subsolution. 

Definition  5.2.  Let  Diea  relatively  open  subset  ofS.  A  C1  function  J  :  B  — »  R  is  said 
to  be  a  subsolution  of  (5.7)  on  D  if 

H(r,—DJ(r))  >0 ,  for  all  r  G  ID.  (5.9) 

Moreover,  J  is  said  to  be  a  solution  to  the  PDE  if  (5.9)  holds  with  equality. 

Recall  the  definition  of  positive  definiteness  given  in  Definition  2.4.  Theorem  5.3 
below  says  that  a  positive  definite  subsolution  of  the  PDE  (5.7)  is  a  local  Lyapunov 
function. 

Theorem  5.3.  Suppose  Condition  2.1  holds.  Let  tt*  G  S°  be  a  fixed  point  of  (1.1),  and 
let  B  be  a  relatively  open  subset  ofS  that  contains  tt*.  Let  J  :  B  — »  R  be  a  C1  positive 
definite  function  that  is  a  subsolution  of  (5.7)  on  B.  Then  J  is  a  local  Lyapunov  function 
for  (1.1)  associated  with  (B,7r*). 

Before  proceeding  with  the  proof  of  the  theorem  we  note  some  basic  properties  of 
the  function  H  introduced  in  (5.8). 

5.4  Properties  of  H 

Lemma  5.4.  Fix  r  G  S  and  a,  a  G  Rd. 

(a)  If  d  —  a  =  cl  for  some  c  G  R,  then  H(r,  a)  =  H(r ,  a). 

(b)  H(r,  •)  is  smooth  and  concave  on  Rd. 
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(c)  Suppose  that  for  each  p  €  S,  F(p)  is  the  rate  matrix  of  an  ergodic  Markov  chain. 
Then  given  r  G  S°,  a  —  a  G  Rd  \  {cl  :  c  G  R}  and  any  p  G  (0, 1), 

H(r,pot  +  (1  -  p)a)  >  pH(r,a)  +  (1  -  p)H(r,a). 


Proof.  The  definition  of  H  in  (5.8)  immediately  implies  part  (a),  and  (b)  follows  since  the 
map  a  —  1  is  smooth  and  convex  for  any  vector  v  G  Rd  and  r^r^r)  >  0  for  all 

x  7^  y,  r  G  S.  To  prove  (c),  fix  r  G  5°  and  a,  a  €  Rd  such  that  w  =  a  —  a  0  {cl  :  c  6  1R} . 
Then  there  exist  x,  y  €  {1, . . . ,  d}  such  that  wx  ^  wy.  Due  to  the  smoothness  and  concavity 
of  H(r,  •),  it  suffices  to  show  that 


Note  that 


dp 2 


H{r,pa  +  (1  -  p)a ) 


cP_ 
dp 2 


H(r,  pw  +  a)  <  0. 


(5.10) 


d^ 

dp2 


H(r,  pw  +  a) 


d?_ 

dp2 


-  E 

x,y£zX\x^y 


(-a-pw,ey-ex)  _  | 


^  T xy(j 


-  Y  K  -  wxf  e(-a-pw’e*-e^rxTxy(r). 

x,y(zX:x=£y 


Since  T  is  ergodic  there  is  a  sequence  of  distinct  states  x  =  xi,  X2,  ■  ■  ■  ,Xj  =  y  such  that 
r XiXi+1(r )  >  0.  Also,  since  wx  7^  wy,  for  some  i  we  have  wXi  7^  wXi+1,  and  so  (5.10) 
follows.  □ 


For  z  G  [0,  00)  let 


Given  r  £  S,  /?  €  Rd,  define 


£(z)  =  z  log  z  —  2  +  1. 


L(r,/3) 


=  inf 
uxyeK+,y^x 


y  '  rx^xy{r)^  ^ 

x,y€.X:Xy^y  ' 


'U'xy 

rxFxy(r) 


C-x)uxy  —  /? 


The  following  lemma  establishes  duality  relations  between  L  and  H. 
Lemma  5.5.  Fix  r  G  S.  For  (3  G  Rd 


(5.11) 


L(r,P)  =  —  inf  \{a,j3)  —  H(r,  a)] , 

aeMd 


(5.12) 


and  for  a  G  Rd 


H(r,  a)  =  inf  [(a,  /3)  +  L{r,  /?)] . 
pend 


(5.13) 


Proof.  Fix  r  G  S,  and  define  H(r,a)  =  —H(r,  —a)  for  a  G  Rd.  Let  V  =  {ev  —  ex  :  x,  y  G 
X,  x  7^  y}.  Then  note  that  for  a  G  Rd, 


H(r,a)  =  Y  Kir, a), 
vev 


(5.14) 


where  for  v  =  ey  —  ex  G  V, 

hv (r,  a)  =  [e^’^  -  lj  rxTxy(r),  a  G  Rd. 
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For  v  €  V,  let  £v(r,  •)  be  the  Legendre  transform  of  hv(r,  •): 

4 (r,  P)  =  sup  [(a,  (5)  -  hv(r,  a)] ,  P  €  Rd. 

a£Rd 

Then  with  A v(r)  =  rxTxy(r)  when  v  =  ey  —  ex, 

/3)  =  |  kv(r)£  ( A^yy)  ^P  =  Ov  for  some  0  >  0, 

(  oo  otherwise. 

From  (5.14),  it  follows  using  standard  properties  of  Legendre  transforms  (see,  e.g.. 
Corollary  D.4.2  of  [7])  that  the  function  L(r,  •)  defined  in  (5.11)  is  the  Legendre  transform 
of  the  function  that  is, 

L(r,P)=  sup  \{a,P)  -  H(r,a)  1, 
aeRd  L  J 

which  is  easily  seen  to  be  eguivalent  to  (5.12).  Finally,  since  H  is  convex  and  continuous 
by  Lemma  5.4(b),  the  duality  property  of  Legendre  transforms  shows  that 

H{r,a)=  sup  [{ce,0)  -  L(r,P)\. 

/3SRd 

This  is  clearly  equivalent  to  the  relation  (5.13),  and  so  the  proof  is  complete.  □ 

We  now  return  to  the  proof  of  Theorem  5.3. 

5.5  Proof  of  Theorem  5.3 

We  begin  by  noting  that,  for  any  r  €  S,  /3(r)  =  rT(r)  satisfies 

H(r,  0)  =  inf  [L(r, /3) \  =  L{r,  P(r))  =  0.  (5.15) 

By  (5.12),  for  any  a  €  Kd 


H(r,a)  <  L(r,P{r))  +  (a,/3(r)). 

We  next  prove  for  any  a  ^  0,  a  €  Ho,  that 

H(r,a)<L{r,P{r))  +  {a,P(r)).  (5.16) 

We  argue  via  contradiction,  and  thus  assume  that  (5.16)  holds  with  equality.  Note  that 
we  must  have  ax  ^  av  for  some  x,  y  €  X,  since  otherwise  a  €  Ho  implies 

0  =  a  ■  1  =  dax,  x  €  X, 

and  then  a  =  0.  Since  ax  ^  ay  for  some  x.  y,  by  Lemma  5.4  (c)  it  follows  that  H(r,  pa)  > 
pH(r,  a)  for  p  €  (0, 1),  and  thus 

H(r,pa)  -  ( pa,P{r )}  >  pL(r,/3(r)). 


Then  from  (5.15) 


0  =  pL(r,P(r)) 

<  H(r,pa)  -  (pa,/3(r)) 

<-  inf  \{a,P{r))  -  H(r,a)\ 

a£Rd 

=  L[r,P(r)) 

=  0, 
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which  is  a  contradiction.  This  proves  (5.16). 

Recall  that  by  assumption  J  is  positive  definite,  and  thus  in  particular  D.J(r)  ^  0 
whenever  r  j=-  ir* .  Applying  (5.16)  to  a  =  —DJ(r)  (recall  DJ(r)  €  Ho),  where  r  =/=■  rr* ,  we 
get 

0  <  H(r,-DJ(r ))  <  ~{DJ{r),  0{r))  +  L(r,  0{r))  =  -(DJ(r), p(r)), 

where  the  first  equality  is  a  consequence  of  the  fact  that  J  is  a  subsolution  of  (5.7)  on 
B,  while  the  last  equality  follows  on  noting  that  L(r,/3(r))  =  0.  Thus  (DJ(r),/3(r))  <  0 
whenever  r/  7r*.  Finally,  note  that 

jtJ (p(t))  =  {DJ(p(t)),p(t)r(p(t)))  =  (DJ{p(t)),P(p{t)))  <  0, 

for  all  0  <  t  <  r  such  that  p(t)  ^  tt *,  where  r  =  inf{t  >  0  :  p(t)  G  Bc},  which  establishes 
the  claimed  result.  ■ 

6  Locally  Gibbs  Systems 

The  PDE  characterization  of  Section  5  gives  an  approach  for  constructing  local 
Lyapunov  functions  for  (1.1).  Although  in  general  explicit  solutions  of  (5.7)  are  not 
available,  there  is  an  important  class  of  nonlinear  Markov  processes  introduced  below 
for  which  solutions  to  the  PDE  (5.7)  can  be  constructed  explicitly,  and  which  generalizes 
the  class  of  Gibbs  systems. 

Definition  6.1.  A  family  of  transition  rate  matrices  {r(r)}rgs  on  X  is  said  to  be  locally 
Gibbs  if  the  following  two  properties  hold: 

(a)  for  each  r  €  <S,  T(r)  is  the  rate  matrix  of  an  ergodic  Markov  chain  on  X,  whose 
stationary  distribution  we  denote  by  ir(r); 

(b)  there  exists  a  C1  function  U  on  S  such  that  for  every  x,  y  €  X,  x  ^  y, 

^ ^  =  exP  (~Dey-emU{r))  ,  (6-D 

where  for  v  G  Ho,  DVU  =  (DU,v). 

The  function  U  is  referred  to  as  the  potential  associated  with  the  locally  Gibbs  family. 
The  following  result  gives  a  local  Lyapunov  function  for  the  ODE  (1.1)  associated 
with  a  locally  Gibbs  family. 

Theorem  6.2.  Suppose  the  transition  rate  matrices  {r(r)}res  are  locally  Gibbs  with 
potential  function  U,  and  let  the  function  J  be  defined  by 

J(r)  =Yjr*  logrx  +U(r),  reS.  (6.2) 

xex 

Then  J  is  a  solution  to  the  PDE  (5.7)  on  S°.  Suppose  in  addition  that  Condition  2.1 
holds  and  J  is  positive  definite  in  a  relatively  open  (in  S)  neighborhood  of  any  fixed 
point  it*  G  5°  of  the  ODE  (1.1).  Then  J  is  a  local  Lyapunov  function  for  the  ODE  (1.1) 
associated  with  rr*  and  the  neighborhood. 

Proof.  Let  {T(r)}re,s,  U  and  J  be  as  in  the  statement  of  the  theorem.  Let  {7r(r)}re,s  be 
the  corresponding  collection  of  stationary  distributions  on  S.  Since  U  is  C1  on  S°  by 
assumption,  J  is  clearly  also  C1  on  S°.  We  now  show  that  J  is  a  solution  to  the  equation 
(5.9).  First  note  that,  due  to  the  locally  Gibbs  condition  (6.1),  for  r  G  <S  and  x. y  G  X, 

*  t ty. 

eDey-exJ(  r)  =  rV7r(r)x 
rx'x(r)y ' 
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Moreover,  since  ir(r)  is  the  stationary  distribution  for  the  Markov  chain  with  transition 
rate  matrix  T(r),  for  any  y  £  X , 

E  n(r)xTxy(r)  =  —ir(r)yTyy(r). 

x£X:x=fiy 

Therefore, 


-H(r,-DJ(r))  =  E  [eD'»- -  l]rxTxy(r) 

x,yEX:x^y 

ryn(r)x  -  rxny(r) 


-  E  - 

x,y£X:x^y 

rv 


7r(r)t/ 


r*„(r) 


=  E  E  -  E r*  E  r* 

y£X  '  y  x£X:x^y  x£X  y£X:y^x 

=  -  E  ryTvy(r )  +  E  r*r**(r) 

yEX  x€.X 

=  o. 


Thus  J  solves  the  PDE  (5.7)  on  S° .  The  result  now  follows  from  Theorem  5.3.  □ 


In  the  rest  of  this  section  we  will  describe  several  examples  that  correspond  to  locally 
Gibbs  systems  and  also  give  an  example  that  falls  outside  this  category,  and  show  that 
for  the  latter  setting  in  some  cases,  the  PDE  (5.7)  can  still  be  used  to  construct  local 
Lyapunov  functions.  In  what  follows  we  will  not  discuss  the  positive  definiteness  property, 
and  instead  refer  to  a  function  that  satisfies  (5.9)  as  a  candidate  Lyapunov  function,  with 
the  understanding  that  if  positive  definiteness  is  added  such  a  function  will  in  fact  be  a 
local  Lyapunov  function. 

The  rest  of  the  section  is  organized  as  follows.  Section  6.1  considers  a  class  of  models 
that  are  a  slight  extension  of  the  Gibbs  systems  studied  in  Section  4.  A  particular  case 
of  locally  Gibbs  that  appears  in  several  contexts  is  introduced  and  discussed  in  Section 
6.2.  Section  6.3  presents  two  examples  of  three-dimensional  systems  which  in  particular 
illustrate  that  Gibbs  systems  are  a  strict  subset  of  locally  Gibbs  systems.  In  Section  6.4 
we  consider  models  with  nearest  neighbor  transitions.  Section  6.5  studies  an  example 
from  telecommunications  [1]  for  which  the  associated  IV-particle  system  has  the  feature 
of  "simultaneous  jumps."  We  show  that  an  explicit  construction  of  a  Lyapunov  function 
carried  out  in  [1]  follows  as  a  special  case  of  Theorem  6.2.  All  examples  in  Sections 
6. 1-6. 5  are  locally  Gibbs  systems.  Section  6.6  considers  an  example  that  demonstrates 
that  the  class  of  models  for  which  a  non-trivial  solution  to  the  PDE  (5.7)  can  be  obtained 
is  strictly  larger  than  that  of  locally  Gibbs  systems. 

6.1  Gibbs  systems 

Recall  the  empirical  measure  functional  rN  :  XN  ->  S  defined  in  (5.1).  Also  recall 
that  throughout  we  assume  r  h*  T(r)  is  Lipschitz  continuous.  We  now  introduce  a 
class  of  models  that  slightly  extend  those  studied  in  Section  4  which,  with  an  abuse  of 
terminology,  we  once  more  refer  to  as  Gibbs  systems. 

Definition  6.3.  Let  I<  :  X  x  IP*  — ►  K.  be  such  that  for  each  x  €  X,  K(x,  •)  =  Kx{-)  is  a 
continuously  differentiable  function  on  Rd.  We  say  a  family  of  rate  matrices  {r(r)}rSs 
on  X  is  Gibbs  with  potential  function  K,  if 

(a)  For  each  r  €  S,  T(r)  is  a  rate  matrix  of  an  ergodic  Markov  chain  with  state  space 
X. 
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(b)  For  each  N  €  IN  there  exists  a  collection  of  rate  matrices  {rJV(r)}rS5  such  that 
VN  —t  r  uniformly  on  S  and  the  N  -particle  Markov  process  XN ,  for  which  the  jump 
rate  of  the  transition 

(x  1 ,  ■  ■  ■  i  Xi— \ ,  Xj,  ,  .  .  .  ,  Xjv)  1  t  (xi ,  .  .  .  ,  X^_l  fUi  1 7  •  •  •  i  Xjv) 
is  r xiV{rN (x)),  is  reversible  with  unique  invariant  measure 

K(xi,rN(x))J  ,  xeXN,  (6.3) 

where  ZN  is  the  normalization  constant: 

l  N 

zN  =  exp  (.^y~i-K'(a;»7'iv(a!)) 
xexN  V  i=i 


N  f  \  x 

7T  (x)  =  —  exp 
ZjN 


-E 


From  Section  4  of  [5]  it  follows  that  the  family  of  rate  matrices  in  equation  (4.3)  is 
Gibbs  in  the  sense  of  Definition  6.3.  Note  however  that  Definition  6.3  allows  for  more 
general  forms  of  rate  matrices  than  (4.3). 

The  following  lemma  shows  that  a  Gibbs  system  is  locally  Gibbs  in  the  sense  of 
Definition  6.1. 

Lemma  6.4.  If  {T(r)}re5  is  Gibbs  with  some  potential  K,  then  it  is  locally  Gibbs  with 
potential  U(r)  =  J2zex  Kz{r)rz. 

Proof.  Since  XN  is  reversible,  the  following  detailed  balance  condition  on  XN  must 
hold: 

7r'Y(x)rx.s/(rJV(x))  =  7rv  (XJJx)  Vyx.(rN  (T^))  (6.4) 

for  every  x  €  XN ,  y  €  X  and  j  €  {1,...,/V},  where  T^x  has  jth  coordinate  value 
equal  to  y,  and  all  other  coordinates  having  values  identical  to  those  of  x.  Since 
rJV(T^x)  =  rAr(x)  +  jyifiy  —  ex:i ) ,  by  (6.3)  and  (6.4),  it  follows  that  for  x  €  XN , 


exp 


K(xi ,  ^(x))  +  K  rJV(x)  +  ^(ey  -  eXj)^ 


L  i= l 


+K  (y,  rN (T^x))  -  K(xj,rN(x )) 


P-  (rJV(x))  =  r-  (r-(^x)).  (6.5) 


Fix  x,  y  €  X,  x  ^  y  and  j  €  IN.  Given  r  €  S,  let  {xi,  X2,  •  •  •  }  be  a  sequence  in  X  such 
that  Xj  =  x  and  with  xA  =  (xi, . . . ,  xjv),  rjv(xiv )  — t  r  as  N  — >  oo.  Since  K  is  continuously 
differentiable,  as  )V  — >  oo 

K  (  rN(xN )  +  —{ev  -ex)j-J2  K(x i>  rN(*N)) 

i—1  '  '  i= 1 

=  N  ]T  (k  UrN(xN)  +  hey  ~  ex))  -  K(z,rN(xN)))  rf  (x*) 

Now,  for  r  €  S ,  define 

n(r)x  =  exp  (■ -Hx(r )) ,  r  e  X, 

Z{r) 
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where  H  was  defined  in  (4.2)  and  Z(r)  is  a  normalization  constant  to  make  7r(r)  a 
probability  measure.  By  sending  IV  — >  oo  in  (6.5)  (with  x  replaced  by  xA ),  using  the 
uniform  convergence  of  TN  to  T  and  the  fact  that  K  is  C1 ,  we  have 

=  ryx(r),  x,  y  G  X,  x  ^  y.  (6.6) 

n{r)v 

This  shows  that  tv  (r)  is  the  stationary  distribution  for  the  rate  matrix  T(r),  and  thus 
verifies  condition  (a)  of  Definition  6.1.  Condition  (b)  holds  because  U  is  C1  due  to  the 
assumptions  on  K,  and  (6.1)  is  verified  by  combining  the  last  display  with  the  fact 
that  U(r)  =  J2zex  Kz(r)rz  and  (4-4)  imply  ~(DU(r),ey  -  ex)  =  log7r(r)y  -  log7r(r)a„ 
x,  y  €  X,  x  7^  y.  Thus,  the  family  {T(r)}r65  is  locally  Gibbs  with  potential  U.  □ 

Given  a  Gibbs  family  of  matrices  {r(r)}r6s,  it  follows  from  Lemma  6.4  that  the 
function  J  :  S  — >  [0,  oo)  defined  in  (6.2)  solves  the  stationary  PDE  (5.7)  and  thus  serves 
as  a  candidate  Lyapunov  function.  Example  4.2  shows  that  in  general  multiple  fixed 
points  of  the  forward  equation  (1.1)  exist  and  that  the  function  J  may  be  positive  definite 
in  the  sense  of  Definition  2.4  for  some  of  the  fixed  points  and  not  positive  definite  for 
others. 

The  locally  Gibbs  condition  is  significantly  weaker  than  the  Gibbs  property.  Indeed, 
it  follows  from  (6.6)  that  Gibbs  systems  satisfy  the  detailed  balance  condition  for  their 
corresponding  rate  matrices,  but  systems  with  the  locally  Gibbs  property  need  not  satisfy 
this  property.  The  simplest  example  is  as  follows.  Let  n  be  the  invariant  distribution 
for  the  ergodic  rate  matrix  T,  and  assume  that  detailed  balance  does  not  hold,  so  that 
it  cannot  be  a  Gibbs  family.  However,  it  is  still  locally  Gibbs,  with  U(r)  =  (r,v)  ,vx  = 
—  log  tvx.  Note  that  in  this  case,  the  proposed  Lyapunov  function  J(r)  in  Theorem  6.2  is 
just  the  relative  entropy  R(r  ||7r).  Example  6.11  below  will  also  illustrate  this  point. 


6.2  A  class  of  locally  Gibbs  systems 

We  now  introduce  a  family  of  ergodic  rate  matrices  {T(r)}r65  that  describe  limits  of 
particle  systems  whose  dynamics  need  not  be  reversible  for  each  N,  (and  hence  may 
not  be  Gibbs  systems),  but  nevertheless  have  a  structure  that  has  some  similarities 
with  Gibbs  systems.  We  show  that  they  are  locally  Gibbs,  and  then  give  two  concrete 
examples  where  they  arise. 

Condition  6.5.  For  a  family  of  transition  rate  matrices  {r(r)}r65  on  X  the  following  two 
properties  hold. 


(a)  For  each  r  €  <S,  T(r)  is  the  rate  matrix  of  an  ergodic  Markov  chain  on  X  with 
stationary  distribution  tv (r). 

(b)  There  exist  R  :  X  x  [0, 1]  — >  R  and  K  :  X  x  ]Rd  — »  1R  such  that  for  each  x  €  X, 
Rx{-)  =  R{x,  •)  is  a  continuous  function  and  Kx(-)  =  K(x,  •)  is  a  C1  function  on  S, 
and  such  that  for  each  r  €  S,  tv (r)  has  the  form 


Hr)x 


exp [-H(x,r)  -  R(x,rx)\ 
Z(r) 


x  €  X, 


(6.7) 


where  H  is  defined  in  terms  of  K  as  in  (4.2),  and  Z(r)  is,  as  usual,  the  normalization 
constant 

z(r)  =  exp [~H(x,r)  -  R(x,rx)\. 

xex 


Note  that  the  Gibbs  systems  from  Section  6.1  satisfy  Condition  6.5  with  R(x,t)  =  1, 
(x,  t)  6  X  x  [0, 1]. 
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Remark  6.6.  Given  a  family  {r(r)}r6s,  suppose  there  exist  R  and  K  as  in  Condition  6.5 
such  that  for  every  x,y  €  S,  x  ^  y. 


exp  [H(y,r)  +  R{y,ry)]Txy(r)  =  exp  [H(x,r)  +  R(x,rx)]Tyx(r).  (6.8) 


Then,  for  fixed  r  €  S,  T(r)  satisfies  the  detailed  balance  conditions  with  stationary 
distribution  n (r)  given  by  (6.7),  and  so  {r(r)}r£5  satisfies  Condition  6.5. 


Lemma  6.7.  Let  {r(r)}r<=,s  satisfy  Condition  6.5.  Then  {r(r)}r£5  is  locally  Gibbs  with 
potential 


U{r )  =  Y 

zex 


R(z,  w)  dw  +  K(z,  r)rz 


(6.9) 


Proof.  First,  note  that  the  conditions  on  R  and  K  ensure  that  U  is  a  C1  function  on  S. 
Thus,  it  suffices  to  verify  equation  (6.1)  of  Definition  6.1,  namely  to  show  that  for  every 
r  €  S  and  x,  y  €  X, 


-log 


ir(r)y\ 

n{r)xJ 


But  this  is  a  simple  consequence  of  the  identity 


d_ 

drT 


R(z,w)dw 


Lzex ' 


=  R(x,rx),  x€X, 


the  fact  that  from  (4.2)  we  have 


d_ 

dr. 


f  K(z,r)rz  J  =  H(x,r),  x  G  X, 
\zex  / 


and  the  definitions  of  n (r)  and  U  in  (6.7)  and  (6.9),  respectively. 


□ 


We  now  provide  two  classes  of  models  that  satisfy  Condition  6.5.  The  first  class  is  a 
system  with  only  nearest-neighbor  jumps. 

Example  6.8.  Let  di,  i  =  1, 2, . . . ,  d  —  1,  and  bi,  i  =  2, 3, . . . ,  d,  be  continuous  maps  from 
S  to  (0,  oo).  Suppose  that  for  r  £  S,  T(r)  is  associated  with  a  birth  death  chain  as  follows: 

r;,i+i(r)  =  ai(r),  i  =  1,2, . . .  ,d-  1, 

-hir),  i  =  2,3, . . .  ,d, 

T itj  ( r )  =0,  for  all  other  i  ^  j. 

As  usual,  set  r^fr)  =  —  so  that  T(r)  is  a  rate  matrix. 

Denoting  by  ir(r)  the  stationary  distribution  associated  with  T(r),  the  {7r(r’)*}i=i . d 

satisfy 

* (r)jttj(r)  =  ir(r)j+1bj+1(r),  for  all  j  =  1, 2, . . . ,  d  -  1. 

The  following  is  a  sufficient  condition  for  Condition  6.5.  Suppose  that  there  are  measur¬ 
able  functions  ipi  :  S  — >  (0,  oo),  i  =  0, . . . ,  d  —  1,  that  are  bounded  away  from  0,  and,  for 
i  =  1, ...  ,d,  continuous  functions  <f>i  :  [0, 1]  — >  (0,  oo),  such  that 

Mr)  =  ^i(r)Mri),  bi(r)  =ipi-1(r)(j>i(ri),  i  =  1, . . . ,  d,  r  €  S.  (6.10) 

Then,  for  j  =  1, ...  ,d  —  1, 

n{r)j  =  bj+1(r)  =  ^j(r)<t>j+1(rj+1)  =  4>j+i(rj+1) 
o,j(r)  ipjirWjirj)  ^(rj) 
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It  follows  that  {r(r)}r65  satisfies  Condition  6.5  with  R(i ,  •)  =  log  </>;(•),/  =  1  ,...,d,  and 
K  =  0.  By  Lemma  6. 7,  it  follows  that  {r(r)}r£5  is  locally  Gibbs  with  potential 


log </>j(w)dw,  u  €  [0,  l],r  €  S. 


Example  6.9.  This  example  can  be  viewed  as  a  generalization  of  the  Glauber  dynamics 
introduced  in  Section  4,  in  which  the  rate  at  which  a  particle  changes  state  can  depend 
both  on  the  state  of  the  particle  and  on  the  fraction  of  particles  in  that  state.  Suppose 
that  we  are  given  R  and  K  as  in  Definition  6.5,  let  H  be  defined  as  in  (4.2),  and  as  in 
Section  4,  let  'k  :  X  x  X  x  S  — >  R  be  given  byi>(x,  y,  r)  =  Hy(r )  —  Hx(r),  x,  y  €  X,  r  €  S, 
and  let  ( a(x ,  y))x,yex  be  an  irreducible  and  symmetric  matrix  with  diagonal  entries  equal 
to  zero  and  off-diagonal  entries  equal  to  either  one  or  zero.  Then,  for  r  €  S,  define 


Fxy(r)  =  exp[-4'(x,  y ,  r)  -  R(x,  rx)]a{x,  y),  x,  y  €  X,  x  /  y. 


Then  the  equality  in  (6.8)  clearly  holds  and  thus  {r(r)}r£s  satisfies  Condition  6.5  by 
Remark  6.6. 

Next  recall  that  Theorem  6.2  shows  that  J(r)  =  U(r)  +  J2 xex  rx  logra.,  with  U  defined 
by  (6.9),  is  a  candidate  Lyapunov  function  for  the  associated  ODE  (1.1).  In  the  present 
example,  consider  the  case  when  there  exists  a  common  Ro  :  [0, 1]  — >  R  such  that 
R(x,-)  =  f?0(-)  for  every  x  €  X  and  K(x,r)  =  —  log(izxR(x,  vx))  for  some  probability 
measure  v  €  V(X)  (and  hence  K(x,r)  does  not  depend  on  r).  Setting  R0(u)  =  ueR°^ 
foru  €  [0, 1],  we  then  have  (up  to  a  constant). 


J(r)  =  J2 

zex 


log 


(Ro(w)\ 

UoK)y 


dw, 


which  is  non-negative  if  Ro  is  non-decreasing.  An  analog  of  this  functional  for  nonlinear 
diffusions  living  in  an  open  subset  fl  of  a  Riemannian  manifold  appears  in  [2],  where 
it  was  shown  to  be  equal  to  the  large  deviation  functional  of  the  so-called  zero  range 
process.  Moreover,  under  the  condition  that  R0  is  strictly  increasing,  it  was  shown  in  [3] 
that  this  functional  (and  a  slight  generalization  of  it,  where  the  logarithm  in  the  integrand 
is  replaced  by  the  derivative  of  a  more  general  C2  function)  serves  as  a  Lyapunov  function 
for  the  associated  nonlinear  PDE. 


6.3  Some  three-dimensional  examples 

Both  classes  of  locally  Gibbs  families  studied  so  far  had  the  property  that  for  each 
r  €  S,  T(r)  is  associated  with  a  reversible  Markov  chain  for  which  the  detailed  balance 
condition  (6.6)  holds.  This  leads  to  two  natural  questions:  (a)  does  every  locally  Gibbs 
family  have  the  property  that  T(r)  satisfies  detailed  balance  for  each  r  €  <S?  (b)  if 
T(r)  satisfies  detailed  balance  for  each  r  €  S,  then  does  it  correspond  to  a  locally 
Gibbs  family?  To  address  these  questions,  we  consider  some  simple  three-dimensional 
examples;  specifically,  Example  6.11  answers  questions  (a)  in  the  negative  while  Example 
6.10  gives  a  partial  answer  to  (b)  by  showing  that  T(r)  may  satisfy  detailed  balance  for 
each  r  €  S,  but  it  may  fail  to  be  a  locally  Gibbs  family  with  any  C2-potential. 

Example  6.10.  Suppose  thatd  =  3,  fix  a;,  bj  >  0 ,  i  =  1,2,  j  =  2,3,  and  let  B  :  S  — >  (0,  oo) 
be  some  given  function.  Forr  €  S,  consider  the  matrix 

(-ai  ai  0  \ 

b-2B(r)  - b2B(r)-a2  a2  ,  (6.11) 

0  b3  -63  J 
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Note  that  for  any  fixed  r  6  S,  the  matrix  (6.11)  corresponds  to  an  ergodic  transition 
matrix  with  stationary  distribution 


7r  (r) 


1 

W) 


(&2&35(r),ai&3,aia2) , 


(6.12) 


where  Z(r)  is,  as  usual,  the  normalization  constant.  Note  also  that  the  detailed  balance 
condition  (6.6)  is  satisfied  for  this  model.  However  as  see  below,  in  general  this  is 
not  locally  Gibbs  system  with  a  C2 -potential.  Let  T  be  the  transition  matrix  in  (6.11) 
when  B(r)  is  replaced  by  1,  and  let  r*  be  the  associated  stationary  distribution.  Fix 
c  =  (ci,  c2,  C3)  €  1R3  and  k  €  IR.  Also,  forr  £  S,  let  T(r)  be  the  matrix  in  (6.11)  with 

B{r)  =  eK{r~r*’c).  (6.13) 


A  simple  calculation  shows  that  if  k  7^  0  and  c2  7^  C3,  there  is  no  C 2  function  U  that 
satisfies  the  equality  in  (6.1)  for  all  x  ^  y  and  r  €  S.  Indeed,  such  a  function  should 
satisfy  for  suitable  real  numbers  a,  /3 


{DU,  er2  —  eri)  =  k  (c,  r)  +  a,  {DU,  er2  —  er3)  =  /3. 


(6.14) 


Taking  second  derivatives  we  see  that 

d2U  d2U  _  d2U  d2U 

dr2  dr2dr\  KC 2’  dr3dr2  dr3dr\ 


and 


d2U  d2U  _ 
dr  2  dr2dr3 

Adding  the  last  two  equations  and  subtracting  from  the  first,  we  have 


d  ( dU  _  dU\ 
dr  1  \  dr  3  dr2  J 


k{c2  -  c3). 


However,  from  (6.14)  the  left  side  equals  0.  Thus  we  must  have  c2  =  c3  or  n  =  0. 
Consequently  when  c2  =/=■  c3  and  0  the  model  is  not  locally  Gibbs.  On  the  other  hand, 
if  c2  =  C3,  one  can  check  that  (6.1)  is  satisfied  with 


U{r)  =  KV\  {r*  -  r,c )  +  log  ri  +  log  r2  +  ^Krf(c  1  -  c2).  (6.15) 

Thus,  when  c2  =  c3,  the  model  is  a  locally  Gibbs  system  with  potential  U. 

Example  6.11.  Letd  =  3  and  forr  €  S,  define  the  rate  matrix  f(r)  by 


f(r) 


—2  r2r3  r2r3  r2r3 
2r\r3  — 4nr3  2nr3 

0  3rir2  — 3rir2 


Then  clearly  rf(r)  =  0. 

As  in  Example  6.10,  for  r  €  S  let  T{r)  be  the  matrix  defined  by  (6.11)  with  B(r) 
given  by  (6.13),  and  let  n{r)  be  the  associated  stationary  distribution  specified  in  (6.12). 
Suppose  that  c2  =  c3  =  c.  It  was  noted  in  Example  6.10  thatn{r)  satisfies  (6.1),  with  U 
as  in  (6.15).  Forr  €  S,  define 

f  (r)  =  f(7r(r)), 

which  takes  the  explicit  form 

(~2a2a2b3  a2a2b3 

2a1a2b2b3B{r)  -4a1a2b2b3B{r) 

0  3aib2b3B{r) 
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Since  rf  (r)  =  0,  7r(r)f  (r)  =  0.  Thus  for  each  r  €  S,  f  (r)  is  the  rate  matrix  of  an 
ergodic  Markov  chain  with  stationary  distribution  7r(r).  Also,  as  noted  earlier,  ir(r) 
satisfies  (6.1).  Thus,  the  family  {f(r)}rS5  satisfies  the  local  Gibbs  property.  However, 
note  that  the  detailed  balance  condition  (6.6),  which  must  hold  for  every  Gibbs  model, 
fails.  Indeed, 

ri2(r)  _  «i  ,  a\  _  7r(r)2 

f2i(r)  2 &2B(r)  ^  &2B(r)  7r(r)i' 

Thus  {f  (r)}rS5  is  not  Gibbs. 

6.4  Systems  with  nearest  neighbor  jumps 

The  nearest  neighbor  model  in  Example  6.8  imposed  certain  symmetry  conditions 
(see  (6.10))  on  the  rate  parameters.  In  the  following  example  we  consider  a  more  general 
family  of  near  neighbor  models  with  certain  monotonicity  conditions  on  the  rates. 

Example  6.12.  Let  X  =  {1,  ...,d}  and  for  r  €  S,  suppose  there  exist  'cost'  vectors 
cl  €  Rd,  i  =  1, . . . ,  d  —  1,  and  continuous  functions  a*  :  R  — )•  (0,  oo)  and  bl  :  R  — »  (0,  oo), 
i  =  1, . . . ,  d,  such  that  for  every  r  €  S,  T(r)  is  the  rate  matrix  of  a  birth-death  chain  that 
satisfies 


rM+i(r)  =  al((r,  c1)),  i  =  1, 2, . . . ,  d  -  1, 

T  i+hi(r)  =  bi+1({r,ci)),  *  =  1,2,...,  d.*-  1, 

Tj  j  (r)  =  0,  for  all  other  i  j, 

and,  as  usual,  set  F u(r)  =  —  J2,  j  jyi  r ij(r),  so  that  T(r)  is  a  rate  matrix.  Let  7r (r)  denote 
the  stationary  distribution  of  the  chain  with  rate  matrix  T(r).  Since  7r(r)r(r)  =  0,  we 
have  for  i  =  1, . . . ,  d  —  1, 


Ar)i+ 1 
n{r)i 


&I+1((r,  c1)) 
o*((r,  c1)) 


n(r,c% 


where  ipl(u)  =  6i+1(w)/a‘(M),  u  €  R. 

Consider  the  specific  case  when  the  cost  vectors  have  the  form  cJ 
j  p  1, . . . ,  d  —  1.  Then  fori,j  =a  1, . . . ,  d  —  1, 

j  j f  1  1  j. 

*+1  *  |  0  otherwise. 

Then  we  claim  that  {r(r)}r£s  is  a  locally  Gibbs  family  with  potential 


(6.16) 


Ed 

k=j+ 1 


&k/ 


(6.17) 


d-l  r{r,ci) 

U(r)  =■  —  /  log  (V’-'(w))  div. 

3=1  Jo 

Indeed,  for  every  r  G  S  and  i  =  1, . . . ,  d  —  1,  using  (6.17) 


d-l 

-Dei+1-eiU{r)  =  ^log  (^((r,^)))  (c|+l  -  4)  =  log  (4>l({r,  c*»)  . 

3=1 

Together  with  (6.16)  this  shows  that  condition  (6.1)  is  satisfied,  and  thus  {r(r)}r£s  is 
locally  Gibbs. 
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6.5  Models  with  simultaneous  jumps 

Weakly  interacting  particles  systems  with  "simultaneous  jumps"  are  described  in 
[1,  8],  For  our  purposes  here  we  need  only  know  that  the  nonlinear  Markov  process 
associated  with  such  models  can  also  be  interpreted  as  the  limit  process  for  an  ordinary 
single  jump  process,  with  an  effective  rate  matrix  that  is  defined  in  terms  of  various 
rate  matrices  used  in  the  definition  of  the  original  process.  We  describe  one  model  with 
simultaneous  jumps  that  arises  naturally  in  telecommunications  and  which  was  studied 
in  [1],  and  show  that  the  associated  family  of  effective  rate  matrices  is  locally  Gibbs. 

In  this  model  there  are  N  nodes,  each  with  capacity  C  €  IN,  and  there  are  M  €  IN’ 
classes,  each  with  parameters  7m  >  0,  and,  in  addition,  a  capacity  requirement 

Am  €  IN.  The  state  of  node  i  is  the  number  of  calls  of  each  class  present  at  that  node  in 
an  N-node  network,  and  thus  the  state  space  takes  the  form 

{M 

X  e  zf  :  XrnAm  <  C 

m— 1 


Let  am(r)  denote  the  average  number  of  customers  in  class  m  under  the  distribution  r : 


am(r)  = 


(6.18) 


It  was  shown  in  Theorem  1  of  [1]  that  the  associated  sequence  of  empirical  measures 
satisfies  pN(-)  =4>  p(-),  as  IV  — »  oo,  where  p(-)  satisfies  the  ODE  (1.1),  with  T  taking  the 
following  form:  for  r  €  S  and  x,  y  €  X,  x  ^  y , 


{Am  +  7 mam{r)  if  y  =  x  +  fm  €  X,  m  =  1, . . . ,  M, 

XmtHm  +  lm)  if  y  =  x  -  fm  €  X,  m  =s  1, . . . ,  M,  (6.19) 

0  otherwise, 

and,  as  usual,  ra,a;(r)  =  —  ^2yex  y^x  T yx(r).  Moreover,  it  is  easily  verified  (see  Proposition 
1  of  [1])  that  for  each  r  €  S,  T(r)  is  the  rate  matrix  of  an  ergodic  Markov  chain  with 
stationary  distribution  7r(r)  given  by 


n{r)x 


1  UZ=i  (Pm(r)T' 

IlMn=l  Xml 


x  =  (xi, . .  -,xM)  €  X , 


where  Z(r)  is  the  normalization  constant  and  p  :  S  — »•  (0,  oo)M  is  given  by 


Pm(r) 


Am  T  7m^m (r ) 
Pm  d-  7 m 


m  =  1, . . . ,  M. 


It  was  shown  in  [1]  that  when  M  =  1  there  is  a  unique  fixed  point  for  the  ODE  (1.1),  but 
when  M  =  2,  A\  =  1  and  A2  =  C  for  C  sufficiently  large,  there  exist  parameters  \m,  pm 
and  7 m  for  which  (1.1)  has  multiple  fixed  points. 

Lemma  6.13.  The  family  of  rate  matrices  {r(r)}r£5  defined  in  (6.19)  is  locally  Gibbs 
with  potential 


M 

U(r)  =  ]T 

m—  1 


^2  [rx  log(xm!) 
_xex 


% mX  x  log  (A 

m  +  / L  m)] 


log  (Am  +  7 mw)  dw 


Proof.  The  function  U  is  clearly  C1 Il  on  S.  For  r  e  S  and  x  =  (xi, . . . ,  xm),  V  =  (2/1, ,  Vm)  € 
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X ,  we  have 


(w(r)x\  -A 

-  log  (  — 7—^  j  =  2_,  l(ym  -  Xm)  log  (pm(r))  +  log(xm!)  -  log(j/m!)] 

\  v  )y  /  m— 1 

M 

=  ^  ^  [(?/m  %m)  (^m  “I-  (^*)  )  (?/m  3?ra)  “h  Twi) 

m=l 

+  log  (a:  m  !)  -  log(ym!)] , 

which  is  easily  seen  to  coincide  with  Dey-ea.U(r),  thus  establishing  that  (6.1)  is  satisfied. 

□ 

By  Theorem  6.2,  it  then  follows  under  positive  definiteness  that  J(r)  =  U(r )  + 
YLxex  r*  l°Srx  is  a  local  Lyapunov  function  for  (1.1).  Using  the  definition  am(r)  = 
J2xex  xmXXf  it  is  easily  seen  that  J  coincides  with  the  Lyapunov  function  g  constructed 
in  Proposition  4  of  [1]. 

Remark  6.14.  Features  of  the  last  example  are  that  the  state  space  X  C  HM  and 
the  rate  matrix  depends  on  r  only  through  the  mean  values  am(r),m  =  1, . . . ,  M.  The 
example  can  be  generalized  slightly.  Indeed,  consider  a  family  of  ergodic  rate  matrices 
{P(r)}re5  on  X  C  Rm,  with  the  property  that  for  each  r  €  <S,  T(r)  has  a  stationary 
distribution  n(r)  of  the  form 

M 

=  n J$(m)(am(r))]  exp  (~H(x,r)),  x€X,r€S , 

m—  1 

where  H  is  the  function  defined  in  (4.2)  for  some  K  :  X  xRd  R,  am  is  defined  by  (6.18) 
and  for  each  m  =  1, ,  M,  4>(m)  :  R  — >  (0,  oo)  is  continuous.  Then,  using  arguments 
exactly  analogous  to  those  used  previously  in  this  section,  one  can  show  that  {r(r)}rf=5 
is  locally  Gibbs  with  potential 


U(r)  =  ]T 

z£:X 


M  /*am(r) 

V  /  log  4>(ml(w)  dw  +  K (z,  r)rz 

_m=  i  J(> 


The  last  example  presented  then  coincides  with  the  case  4 >m(w )  =  Am  +  7mw,  w  €  R, 
and  for  r  €  S 


M 

K(x,r)  =  H(x,r)  =  ^  (xm  \og{pm  +  7m)  +  log(xm!)) . 

m—  1 

Remark  6.15.  Another  example  of  a  model  with  simultaneous  jumps  is  the  model  of 
alternative  routing  in  loss  networks  introduced  in  Gibbens,  Hunt  and  Kelly  [9],  It  can 
be  shown  that  the  family  of  jump  matrices  {r(r)}res  associated  with  this  model  is  not 
locally  Gibbs  with  anyC2  potential  U .  This  may  explain  why  this  problem  has  withstood 
analysis  for  more  than  a  decade.  It  is  an  interesting  open  problem  to  see  if  the  PDE 
characterization  introduced  here  can  be  used  to  construct  Lyapunov  functions  for  this 
model  and  related  ones. 

6.6  A  candidate  Lyapunov  function  for  a  model  that  is  not  locally  Gibbs 

The  example  in  this  section  demonstrates  that  the  class  of  models  for  which  explicit 
non-zero  solutions  of  (5.7)  can  be  found  is  larger  than  that  of  locally  Gibbs  models.  Let 
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d  =  3,  and  for  r  €  S  define  the  rate  matrix  T(r)  by 

/  -ai(r)  oi(r)  0  \ 

r(r)  =  b2(r)  -(a2(r)  +  b2(r))  a2(r)  , 

V  0  h{r)  ~b3(r)  ) 

where  a\  and  a2  are  measurable  functions  from  S  to  (0, 1)  and  b2,  b3  are  given  as  follows. 
Let  tf>  :  [0, 1]  — »  (0, 1)  be  a  continuous  function  that  is  bounded  away  from  0.  We  set 

62 (r)  =  (!  +  (r2  ~  r3ip(r3))a2{r))  «i(r),  b3(r)  =  tp{r3)a2(r)  (1  +  (r2  -  rr)ai(r)) . 


Note  that  for  each  r  €  S,  T(r)  is  an  ergodic  rate  matrix  and  the  corresponding  unique 
invariant  measure  7r(r)  satisfies 


"TT  =“TT  =  ^  +  ““  ^(r3))o2(r)) , 

n{r)2  ai{r ) 

^TT  =  =  ^  +  ““  ri)ai(r))  • 

n(r)3  a2(r ) 


Since  ai,  o2  are  arbitrary  functions,  there  may  be  no  C1  function  U  for  which  equation 
(6.1)  is  satisfied,  and  so  the  family  {r(r)}r6s  is  not  locally  Gibbs  in  general. 

Define 

rr3 


U(r)=  /  logi l>(x)dx,  r  £  S, 

Jo 

and  let  J  be  defined  through  (6.2).  Then,  as  shown  below,  J  satisfies  the  PDE  (5.7)  on  S° 
and  hence  is  a  candidate  Lyapunov  function.  Indeed,  note  that 


Thus, 


'  log(^) 

if  (y,  x) 

=  (2,1), 

logpt^) 

if  (y,  x ) 

=  (3,1), 

,  logPtir3)) 

if  (y,x) 

=  (3,2). 

-H(r,  - DJ(r ))  =  (r2  -  ri)ai(r)  +  ( r3ip(r3 )  -  r2)a2(r)  +  (n  -  r2)b2{r) 

+ (f2  ~,^.(r,))  h(r) 
ip(r3) 

=  (r2  -  n)ai(r)  +  ( r3ip(r3 )  -  r2)a2(r) 

+  (n  -  r2)  (1  +  (r2  -  r3ip(r3))a2{r))  ai(r) 

+  ^  ^(r^)3^^(r3)a2(r)  (X  +  (r2  -  n)ai(r)) 

=  0. 
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